Memorandum 


To: Interested Parties 

From: Mark Johnson, NC Superintendent of Public Instruction 

Re: Public Records from the Read to Achieve RFP 

There is a great deal of misinformation in public forums regarding the Read to Achieve 
diagnostic selection process. DPI cannot release every detail of the procurement process until the 
process is complete (i.e. until after the protest is decided). Unfortunately, that means the public 
records released now might not present a full picture of the process. It is our hope, though, that 
these public records help to eliminate some of the misinformation. 


Public Records Context 


A review of evaluation committee notes and an internal update presentation included in 
these public records will reveal misstatements of facts put forth by members of the evaluation 
committee. Many of these misstatements were clarified and corrected later in the process, such as 
statements regarding dyslexia screening. In the case of the update presentation from December of 
2018, the slides were not updated to correct misstatements of fact, missing information, or the 
Phase 1 rankings based on such misstatements because the presentation was never publicly 
delivered. The Contract Award Recommendation and presentation to the State Board of 
Education contain correct information. 


Statement from Superintendent Johnson 


“Istation is the best reading diagnostic tool for North Carolina, and I believe using 
Istation will yield quality data that will better support success for our students, meeting students 
where they are and helping them grow, while also reducing the time teachers must spend testing 
students. DPI and the State Board adhered to all laws, rules, and policies during this procurement 
to ensure fairness and objectivity. We are excited about the end result of a partnership with 
Istation to support students and teachers across North Carolina.” 
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Screen an entire class 
on average in about 
- 25-45 minutes! 


Active monitoring 
is vital for 
accurate results 



ISIP Early 
Reading 


Grade 


Kindergarten 


1 st Grade 


2 nd & 3rd Grade 


Subtest 


Listening Comprehension 
Phonemic Awareness 
Letter Knowledge 
Vocabulary 

Phonemic Awareness 
Letter Knowledge 
Vocabulary 
Alphabetic Decoding 
Comprehension 
Spelling 

Connected Text Fluency *(Maze/Cloze Passage) 
Oral Reading Fluency 

Vocabulary 

Comprehension 

Spelling 

Corrected Text Fluency * (Maze/Close Passage) 
Oral Reading Fluency 


Text Fluency subtest not included in overall ability 


Valid and Reliable Measures 


Drive Instruction Based on the Science of Reading 






















Implementation Plan 

Use Istation at the beginning of the 2019-2020 school year (with delay in metrics) 
Gather data to become familiar with the assessments during the fall 
Use data to inform instruction 

Delay the use of data to measure growth for EVAAS until MOY benchmark 
Use first official benchmark in the winter (MOY) and end of year benchmark (EOY) 
for EVAAS purposes 

Train all teachers by start of school by continuing aggressive schedule including: 

- in-person workshops 

- on-demand webinars 

- learning modules (podcasts) 

- technical assistance 

- ongoing support from the K-3 Literacy team 


Impact of Measuring Growth Using Middle of 

Year (MOY) to End of Year (EOY) 

Since EVAAS growth is a relative measure of performance, there is no predetermined level 
a student must reach to show growth. 

As long as the measurement period for all participants is roughly the same, the model will 
yield a valid estimate of growth for a teacher relative to peers in the same grade and 
subject. 

Teachers in the state will not be disadvantaged by the shorter measurement period 
because we are comparing the progress one teacher's students made to all the other 
teachers' progress with their students (in the same grade and subject) in the same amount 
of time. 

The State has always measured kindergarten growth this way. 

Growth for third grade is not based on the diagnostic assessment and will continue to be 
measured BOG/EOG for EVAAS. 


Istation and North Carolina Training Schedule 2019-2020 

Dates for technical and educator webinars will be 
conducted in a live Q&A environment (recorded and 
available for on demand viewing) 

Planning and meetings to prepare for launch of Istation across 

North Carolina 

Release of Recorded Online Module for educators to view 
brief course, complete quiz, and attain certificate of 
completion (Podcast) 

Face to face training provided by Istation for chosen campus leader 
(regional based interactive workshop to bring back to school level) 

Onsite professional learning for DPI Stakeholders 


January (Traditional) 
First benchmark window 
for 2019-2020 opens for 
EVAAS purposes (MOV) 
(December for YR) 
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Training 

Opportunities 

Face to Face - Webinars - Podcasts 


Informational Webinar for 
Technical Contacts 

This Istation webinar focuses on technical specifications, 
deployment, and integrations. Technical contacts should join 
this one-hour live session. 

Webinar times are listed as Eastern Daylight Time (EDT). 



Informational Webinar for Educators 

This Istation webinar focuses on the product, educational 
best practices for district, campus, and classroom use. Learn 
more about the education side of Istation. 

Webinar times are listed as Eastern Daylight Time (EDT). 


Istation's Indicators of Progress, 
Early Rea ding (ISIP-ER ) 

• Sophisticated, Computer Adaptive Testing (CAT) Syster 

• Provides Benchmark & Continuous Progress Monitoring 

• Assesses students in critical domains of reading 
throughout academic years 

• Game-like & engaging environment 

• Award winning, valid & reliable 

• Saves teachers' instructional time! 




Podcasts (Online Modules) begin July 15th 


On-Site Training - Register Now 


Durham - Tuesday, July 2, 2019 - Year Round Schools 
Registration will close July 1 st at 4:00 p.m. 

Greensboro - Monday, July 8, 2019 - Year Round Schools 

Registration will close July 7 th at 4:00 p.m. 

Morganton - Monday, July 22, 2019 - Region 7 Northwest 

Registration will close July 21 st at 4:00 p.m. 

Gastonia - Tuesday, July 23, 2019 - Region 6 Southwest 
Registration will close July 22 nd at 4:00 p.m. 

Gastonia - Wednesday, July 24, 2019 - Region 6 Southwest 

Registration will close July 23 rd at 4:00 p.m. 

Greensboro - Wednesday, July 31,2019- Region 5 Piedmont-Triad 

Registration will close July 3Cf h at 4:00 p.m. 

Greensboro - Thursday, August 1,2019 - Region 5 Piedmont-Triad 

Registration will close July 31 s ' at 4:00 p.m. 

Asheville - Friday, August 2, 2019 - Region 8 Western 

Registration will close August I s1 at 4:00 p.m. 

Fayetteville - Monday, August 5, 2019 - Region 4 Sandhills 
Registration will close August 4 m at 4:00 p.m. 

Durham - Tuesday, August 6, 2019- Region 3 North Central 

Registration will close August 5 lh at 4:00 p.m. 

Durham - Wednesday, August 7, 2019 - Region 3 North Central 

Registration will close August 6 th at 4:00 p.m. 

Wilmington - Thursday, August 8, 2019 - Region 2 Southeast 
Registration will close August 7" 1 at 4:00 p.m. 

Greenville - Friday, August 9, 2019 - Region 1 Northeast 
Registration will close August 8 th at 4:00 p.m. 


registration closed 


Register 


Register 

Register 

Register 

Register 

Register 

Register 

Register 

Register 

Register 

Register 

Register 


Note: Upon completion of training, participants will receive a certificate which can be used to earn CEU credit 







































Getting Started with Istation Roll-out Plan 


Summer 2019 (June, July, August) 

•Istation enrollment and deployment activated in schools 
•Live webinars hosted 
•In-person regional trainings hosted 
•On-going implementation support 

•Districts can begin using Istation as early as July, but it will not count in metrics 


i&l 


Istation 


lr NCDPI 


Fall 2019 (September, October, November) 

•Students take station's ISIP assessment to begin to learn from the program 
•Additional in-person regional trainings hosted 
•Additional live webinars hosted 

•Fall is a "getting started" learning opportunity: Data will not feed into EVAAS 

Winter 2019-2020 (December, January, February) 

•Ongoing progress monitoring continues 
•Ongoing training continues 

•JANUARY: The first benchmark window for 2019-2020 opens for EVAAS purposes (MOY) * 


Spring 2020 (March, April, May) 

_ . .. * Traditional Calendar 

•Ongoing progress monitoring continues (December/Apmfor yr) 

•Ongoing training continues 

•MAY: End-of-year benchmark window for 2019-2020 opens for EVAAS purposes (EOY) * 


Istation Measures for Dyslexia Screening 


Kindergarten 

• Automatically screened upon login 

• Additional subtest 

* alphabetic decoding * 

• Normal ISIP kindergarten subtests 

* listening/language 
comprehension 

* phonological and phonemic 
awareness * 

* letter knowledge * 

* vocabulary 


1st Grade 

• Automatically receive the relevant 
subtests based on the initial 
screening 

* phonological and phonemic 
awareness * 

* letter knowledge * 

* vocabulary 

* alphabetic decoding * 

* reading comprehension 

* spelling 





Istation Measures for Dyslexia Screening 


2nd Grade 

• Automatically screened upon login 

• Additional subtests 

* alphabetic decoding * 

* letter knowledge * 

* phonological and phonemic 
awareness * 

• Normal ISIP subtests for 2nd grade 

* vocabulary 

* reading comprehension 

* spelling 

* text fluency 


3rd Grade 

• Automatically screened upon login 

• Additional subtests 

* alphabetic decoding * 

* letter knowledge * 

* phonological and phonemic 
awareness * 

• Normal ISIP subtests for 3rd grade 

* vocabulary 

* reading comprehension 

* spelling 

* text fluency 





Next Steps - Read to Achieve 


DPI is purchasing the diagnostic assessment directly, as before 

DPI has Read to Achieve funding available for implementation of the new 

diagnostic assessment: 

• Devices - each classroom should have sufficient devices to implement 
a work station approach (4 is recommended guideline) 

• In addition to this funding, DPI has iPads available to distribute 

• Accessories - for example, each device should be equipped with a 
headset with microphone 

• Training expenses such as summer stipends 


Next Steps - PRC 085 Allotment Policy 

Manual Update 

Policy was updated at June SBE meeting 

- to remove one-time allotments from 2017-18 not continued in 2018-19 

- to provide a clean slate for 2019-20 changes once assessment contract awarded 
Propose update to current version of policy based on feedback from local 
superintendents in June (current version only provides for device refresh) 
Empower districts to make locally-informed choices on K-3 Literacy spending 
while maintaining State-defined parameters aligned with Read to Achieve 
law, including anticipated 2019 amendments 

Update to add the following allowable expenditures: 

- K-3 literacy aligned instructional supports 

- Training and personnel to support K-3 literacy instruction 
Continue allotment of funding based on K-3 ADM 


Next Steps - Read to Achieve 


Operational & Policy Steps 

• Update Read to Achieve Guidebook 

• Annual updates to Local Alternative Assessment list 

• Grade level expectations for proficiency 
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READ TO ACHIEVE 

K-3 Literacy Division 
NC Department of Public Instruction 


Istation and Read to Achieve in North Carolina Video 











nUEu 

Public Schools of North Carolina 


Questions? 


Dr. Tara Galloway 
K-3 Literacy Director 



o 


NC 


6 


O 


DIT 


6 


Roy Cooper 

Governor 


Eric Boyette 

Secretary of Information Technology 
State Chief Information Officer 


Contract Award Recommendation 


To: Andrea Pacyna 

Deputy Chief IT Procurement Officer 
Department of Information Technology 

From: Tymica Dunn 

Procurement Chief 
Department of Public Instruction 

Date: June 7, 2019 

Subject: Contract Award Recommendation 

Read to Achieve Diagnostics - Requisition # - RQ20680730, DIT File #300042 

Reference #: Request for Negotiations 40-RQ20680730A, DIT File #300042 

Enclosed for your review and approval is the award recommendation for Requisition # RQ20680730. 

Bids received pursuant to RFN #40-RQ20680730A have been reviewed and an Evaluation Committee hereby requests the Statewide IT 
Procurement Office to award the contract, as follows: 

Description: Read to Achieve Diagnostics - Software as a Service 

Recommended Vendor: Imagination Station Inc., dba, Istation 
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Cost: 


$8,405,820 for 3 years 


Contract Term: Two (2) years plus 1 (one) 

year optional renewals at the discretion of the State 
Project Name and Number: Read to Achieve Diagnostics - 2018 

DIT file #300042 

Thank you for your assistance. If additional information is required, please do not hesitate to contact me. 

cc: Evaluation Committee 

Patti Bowers, DSCIO 
Glenn Poplawski, DSCIO 
Kathy Bromead, PMA 
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Section 1: Introduction 

The North Carolina Department of Public Instruction posted Request for Proposal number 40-RQ20680730A to the North Carolina 
Interactive Purchasing System on September 6, 2018. A total of four (4) bids were received; however, the evaluation committee 
could not reach a consensus and deemed it most advantageous to the State to cancel and negotiate with sources of supply. NCDPI 
requested and received approval from the DIT DSCIO/Chief Procurement Officer to negotiate. 

Request for Negotiations were sent to Amplify and Istation on March 28, 2019 and negotiation meetings were conducted on April 
11, 2019 with both vendors at North Carolina Department of Public Instruction. 

DSCIO/Chief Procurement Officer 

The purpose of this award recommendation and the resulting contract award is to identify a vendor best qualified to offer services 
for Read to Achieve Diagnostic Software as a Service solution (RtAD) to meet NCDPI's obligations under state law, N.C.G.S. 115C- 
83.1, et. seq. 

North Carolina state law requires kindergarten through third grade students to be assessed with valid, reliable, formative and 
diagnostic reading assessments. NCDPI is obligated to adopt and provide these developmentally appropriate assessments. The solution 
must assess student progress, diagnose difficulties, inform instruction and remediation, and yield data that can be used with the 
Education Value-Added Assessment System (EVAAS). 

Section 2: Evaluation Committee 


Name 

Title/Agency 

Participation 

Level 

Berry, Erika 

Senior Policy Advisor, NCDPI 

Decision Maker 

Craver, Nathan 

Digital Teaching and Learning Consultant, NCDPI 

Decision Maker 

Karkee, Thakur 

Psychometrician, NCDPI 

Decision Maker 

Shue, Pam 

Deputy Superintendent of Early Education, NCDPI 

Decision Maker 

AlHour, Julien 

Director - Architecture, Integration, & Quality 
Assurance, NCDPI 

SME 

Dunn, Tymica 

Purchasing Section Chief, NCDPI 

Procurement 

Officer 
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Gossage, Chloe 

Chief Strategy Officer, NCDPI 

SME 

Strong, Melissa 

State Board of Education Attorney 

SME 

Viswanathan, Srirekha 

Project Manager, NCDPI 

Project Manager 


Role Definitions: 


Decision Maker: Key business stakeholders evaluating the bid 

responses. 

Project Manager: Overall responsibility includes successful 

initiation, planning, design, execution, 
implementation, and closure of a project. 


Subject Matter Expert Person who is an authority in a particular 
(SME) technical area pertaining to the procurement 


Voting 


Non-Voting 


Non-Voting 


Section 3: Evaluation Criteria / Methodology 

The selection process was conducted using the "best value" methodology authorized by N.C.G.S. §§143-135.9 and 143B-1350(h). The 
evaluation committee met as a group and evaluated the responsive proposals. 

The evaluation criteria listed below is in the order of importance: 


Evaluation Criteria 


Cost 

Vendor Financial Stability 
Formative and Diagnostic Assessment 
Personalized Learning 
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Section 4: Timeline 


Date 

Milestone 

March 21, 2019 

RFP Cancellation 


Notifications sent to vendors, Request to Negotiate 


Review Period 

March 27, 2019 

RFP proposals were extended to June 29, 2019 - Clarification 1 

April 11, 2019 

Negotiation Meeting with vendors 

April 17, 2019 

Clarification issued to vendors - Clarification 2 

April 23, 2019 

Clarification response received and shared with evaluation team 

April 25, 2019 

Evaluation Committee meeting and discussion of proposal 
strengths and weakness 

May 3, 2019 

Clarification issued to vendor - Clarification 3 


Clarification response received and shared with evaluation team 

May 15, 2019 

Clarification issued to vendor - Clarification 4 


Clarification response received and shared with evaluation team 

June 4, 2019 

Best and Final Offer (BAFO) 

June 6, 2019 

Award Recommendation 


Section 5: Evaluation of Bid Submission 

Proposal response from the following two vendors were considered for further negotiations: 


mber 

Company Name 

Address 

1 . 

Amplify Education Inc. 

55 Washington Street, Suite 800, Brooklyn, NY 11201 

2. 

Imagination Station 
dba, Istation 

8150 North Central Expressway, Suite 2000, Dallas, TX 75206 
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Section 6: Vendors 

Listed below is a synopsis of each proposal submitted based on the criteria defined in Section 3. 

A. Evaluation Criteria 

"Best Value" procurement method authorized by N.C.G.S. §§143-135.9 and 143B-1350(h) has been used for this evaluation. A one 
step source selection was used. The proposals were objectively evaluated using the evaluation criteria described below. 

The evaluation team members did their due diligence and issued clarifications for each proposal before meeting the vendors on April 
11, 2019. Strengths and weaknesses were discussed during the evaluation meeting on April 25,2019. 

The following evaluation criteria was used to determine strengths and weakness - 

1. Cost 

2. Vendor Financial Stability 

3. Formative and Diagnostic Assessment 

4. Personalized Learning 

B. Cost 

The strengths and weaknesses identified by the Evaluation team for the responsive vendors are summarized in the tables 
below. 


Cost 

Vendor 

Strengths 

Weakness 

Amplify 

No strengths noted. 

1. Amplify submitted two cost offers - one for 

assessment only at $4,312,210 (Year 1), $3,895,210 (Year 
2), $3,883,760 (Year 3) totaling $12,102,096.08 another 
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one for personalized and blended approach to 
learning at $11,948,912.75 (Year 1), $10,934,412.75 
(Year 2) and $10,922,962.75 (Year 3) totaling 
$33,806,288.25. The assessment only cost which was 
considered for this proposal review is significantly 
higher than Station's assessment only tool. 

2. The assessment cost of $8.00 per student is higher 
than that of Istation and does not include online 
assessments nor remote student or parent access. 

3. This cost does not include teacher lessons. 

4. The assessment is not automated and requires 
teacher intervention by reading the tests aloud and 
takes away significant classroom time from teaching. 

5. Professional Development cost for year 1 is 556,650; 
however, is limited to training Master Literacy 

Trainers and NCDPI Consultants. The proposal 
response did not adequately include strategies for 
ensuring consistent scoring to evaluate training 
effectiveness. 

Istation 

1. Istation submitted two cost offers one for the assessment 
component only and one for the both the assessment and 
curriculum components. The cost for the assessment was 
$2,751,940 (Year 1) $2,751,940 (Year 2) $2,751,940 (Year 

3) totaling $8,255,820. For both the assessment and 
curriculum was $9,934,813 (Year 1), $9,934,813 (Year 2), 
$9,934813 (Year 3) totaling $29,804,438. 

2. The assessment cost of $5.70 per student is less expensive 
than Amplify and includes more features such as 3,000 
teacher directed lessons, remote student and parent 
access to Station's iPractice. 

1. Solution is not compatible with screen readers or 
keyboards and will cost extra to ensure compatibility. 
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3. $76,103 for professional development offers 22 onsite 
trainings, 14 recorded live webinars and 10 virtual teacher 
trainings annually, in addition the vendor will provide up to 
5 additional onsite and 10 recorded webinars annual at no 
additional cost. 

4. Vendor will provide additional professional development 
beyond these allowances at a rate of $5,800.00 per day of 
professional development and $550.00 per webinar. 

5. The cost for Professional Development also covers the 
logistics which includes securing learning facilities, paying 
the cost to host the training, coordinating training dates, 
communication to participants etc. 


C. Vendor Financial Stability 

The strengths and weaknesses identified by the Evaluation team for the responsive vendors are summarized in the tables 
below. 


Vendor Financial Stability 

Vendor 

Strengths 

Weakness 

Amplify 

NCDPI Financial Director finds no going concern. 

None 

Istation 

NCDPI Financial Director finds no going concern. 

None 


D. Formative and Diagnostic Assessment 

The strengths and weaknesses identified by the Evaluation team for the responsive vendors are summarized in the tables 
below. 
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Formative and Diagnostic Assessment 


Vendor 


Strengths 


Weakness 


Amplify 

1. Assessment covers all five areas of early 
literacy which is mandated by law. The 
service has the capability to appropriately 

assess K-3 students. 

2. Amplify Service has enough item pool for 

20 assessments (i.e., number of items 
that are aligned to NC standards which 
will be enough for 20 tests). It is also to 
be noted that Schools have three tests per 
grade level for this age group. 

3. The reports are easily understandable. 
Home Connect Letters for parents is clear. 
There are multiple reports for teachers 

about instruction and areas that need 

intervention. 

1. Benchmarking and progress monitoring per student per 
grade level consumes a lot of time and requires excessive 
teacher involvement to manually administer and enter test 
results. The fixed form manual test takes more time testing 
to find where the students are at. This takes away significant 
instructional time. 

2. The $8 option is not adaptive i.e., it does not measure 
student's exact level of achievement. It was difficult to 
gauge from the proposal response how the service 
adapts when students gain mastery. 

3. The fixed form tests don't always provide feedback on 
the student's exact level of achievement which brings 
to question the effectiveness of the data driven 
instructional support. 

Istation 

1. Adaptive assessment (also known as 
Computer Adaptive Assessment) allows 
students to reach their full potential. This 
assessment measures student's mastery 
with the minimal amount of teacher time. 

2. The aggregate reports for teachers are 
easy to read and interpret. 

3. Istation has enough item pool for 10 
assessments (i.e., number of items that 
are aligned to NC standards which will be 

None 
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enough for 10 tests). It is also to be noted 
that Schools have three tests per grade 
level for this age group. 


E. Personalized Learning 

The strengths and weaknesses identified by the Evaluation team for the responsive vendors are summarized in the tables 
below. 



Personalized Learning 

Vendor 

Strengths 

Weakness 

Amplify 

1. Personalized Learning was only offered in the 

Alternate Cost proposal which came with 
increased pricing. 

1. The basic cost proposal offered does not have all 
aspects of personalized learning and is not computer 
adaptive. 


2. Progress Monitoring when a student is 
identified as at risk for achievement, is at 
individual skills level. 

3. Amplify offers a dyslexia component. 

2. Progress Monitoring for students at risk requiring 
intervention takes up a lot of time for teachers. The 
basic assessment solution option is not computer based 
it is takes away significant instruction time from 
teachers and the reliability and validity of results vary 
significantly. 

3. Home Reading is not included in the bid offering. This 
limits the ability for students to have access to 
resources outside of school which limits their learning 
and the participation from parents. 
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Istation 


1. The assessment is computer adaptive and caters 
to the individual student's need. 

2. The time for assessment offered by Istation is 40 
minutes/student and is fully online (i.e., teacher 
can work with other students in class while a 
group of students are taking the assessment). 
Amplify's assessment is 45 minutes/student on 
the low end and requires teachers to spend time 
with the students while they are being assessed. 
The reduced assessment time and the fact that 
the teacher does not have to be with students 
who are being assessed (using the computerized 
model) allows teachers more time to support 
student's individual needs. 

3. Istation allows students see their own academic 
need and take responsibility for their learning by 
providing feedback after each subtest. This 
feedback is available to students, parents and 
teachers. Further students are allowed access 
outside of school. They can personalize their 
learning by choosing games and activities to 
further enhance their learning. 
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1. Although Istation stated that their assessment can be 
used to screen for dyslexia, the vendor does not have a 
separate dyslexia component at this time. 
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Section 7: Finalist Vendor(s) 


NCDPI entered into negotiations with both vendors. Each vendor was given the opportunity to present their assessment solution 
and how it would best meet the needs of the department. 

Clarification 1 was issued to both vendors extending their RFP bid submission as the proposal response was used in the negotiation 
process. 

Clarification 2 was issued to both vendors prior to the negotiation meeting. The question provided in this request were focal points 
during the meeting. This clarification request also gave the Evaluation Team some guidance and understanding with both vendor 
offering. After the negotiation meeting held on April 11, 2019 the team unanimously agree to continue further negotiation efforts 
with Istation. 

Clarification 3 Istation was asked by NCDPI to provide the cost of both the assessment and curriculum. This request was to compare 
the Alternative Cost proposal 2 submitted by Amplify which included the curriculum portion. After reviewing Istation's submission 
the team agreed to go with only the assessment portion which is required in legislation. While there was in interest in the 
curriculum offering it is not required in the law. 

Clarification 4 was issued to negotiation on the Terms of Use and Privacy policy that Istation has in place. NCDPI's legal team 
negotiated the language that was provided by Istation. Istation was in agreement and signed the clarification giving the department 
permission to incorporate in in the final contract offering. 

While Amplify was able to submit an offer to satisfy the agencies needs it was not cost effective. As the incumbent the progress 
made by students in reading is not significant. The effectiveness of the data driven instructional support is questionable. The 
current test scores does not support the inflated cost offered by Amplify. 

Istation provided a solution that was robust, cost effective, offered additional enhancements that were required, and met the 
business needs of NCDPI. While Istation's dyslexia component may be missing key measures, the service substantially conforms to 
the requirements specified under N.C.G.S. 115C-83.1, which is the primary obligation of this procurement. 

Negotiations were issued to Istation and memorialized in the BAFO # 40-20680730A dated June 4, 2019 in which Istation agreed to 
the following change in specifications: ADA Compliance high contrast reports. Voice Recognition Software, Onsite Training and 
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Recorded Webinars, Growth Calculation, Summer Reading Camps, Customizations and Enhancements, BAFO Cost, as well as 
modifications to the Istation Terms of Use and Privacy Policy which comprise the License grant and agreement for the State's use of 
the Istation Resources. 

IStation also completed the Vendor Security Assessment Guide (VRAR) that was reviewed and approved by NCDPI and DIT technical 
teams. 

Section 8: Award Recommendation 

The Evaluation Committee has determined that Istation's bid substantially conforms to the specifications and requirements of the 
law and therefore, recommends award RFP No. 40-RQ20680730A to Imagination Station Inc. (Istation) in the amount of $8,405,820 
(Year 1 - $2,751,940, Year 2 - $2,751,940, Year 3 - $2,751,940) for 2 years with the option of one (1) additional one (1) year 
renewals. 

Section 9: Supporting Documentation 

The following supporting documents that reflect the vendor selection are included: 

1. Bid Response - 

2. Clarification documents- 

3. Signed BAFO document 

4. Hosting Exception and Privacy and Threshold Analysis (approved by DIT) 
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EVALUATION COMMITTEE MEMBERS 
CONFIDENTIALITY AGREEMENT 
For 

RFP # 40-RQ20680730 - Read to Achieve Diagnostics - Software as a Service (RtAD- 
SaaS) 


Pursuant to North Carolina’s Administrative Code 09 NCAC 06B.0103, all information 
and documentation (verbal and written) relative to development of a contractual document is 
deemed “confidential” and shall remain confidential until successful completion of the 
procurement process. 

Therefore, Evaluation Committee Members (both voting and contributing advisors) are 
required to keep all comments, discussions, and documentation confidential until a notification 
of award has been made by the Issuing Agency for this solicitation. By participating in this 
Evaluation Committee, you agree to not divulge any information to an unauthorized person in 
advance of the time prescribed for its authorized release to the public. This includes co¬ 
workers, supervisors, family, friends, etc. 

If it is discovered that there has been a breach of confidentiality by a member of this 
Committee, he/she will be immediately excused by the Committee Chair until further notice. 

The solicitation may be cancelled and a new solicitation may be issued with a new Evaluation 
Committee. 

In addition, the issue will be referred to the employee’s department director or agency 
head. Department directors or the heads of autonomous agencies shall be responsible for the 
preliminary examination and investigation of reports from employees of any violations which 
compromise the procurement process. If, following a preliminary examination and investigation, 
the department director or agency head finds evidence of a violation or finds that further 
investigation is warranted, a report shall be submitted to the respective Human Resources 
Office for potential disciplinary action. 

By signing below, I certify that, as a member of this Evaluation Committee, 

I will keep all comments and discussions, preliminary / working evaluation 
notes, and all other information (verbal and written) regarding the above 
referenced solicitation, confidential until after a notification of award has 
been made by the Issuing Agency. 


Samiel Fuller 


Signature 


Date 



EVALUATION COMMITTEE MEMBER S 
STATEMENT REGARDING 
CONFLICT OF INTEREST AND DISCLOSURE 
For 

RFP # 40-RQ20680730 - Read to Achieve Diagnostics - Software as a Service (RtAD-SaaS) 

The following organizations have submitted a bid proposal and response to the above solicitation: 


1) Amplify Education Inc. 

2) Curriculum Associates (i-Ready) 

3) Imagination Station Inc. (IStation) 

4) NWEA 










Each member involved in the evaluation process must verify that he / she has no personal, financial, 
business or other conflicts of interest, with regard to this procurement and his / her official duties as an evaluator. 

North Carolina General Statute § 143-58.1 prohibits unauthorized use of public purchase(s) or contract 
procedure for private benefit. Therefore, by signing this statement, you certify that neither you nor members of 
your immediate family currently have or expect to gain, any personal, financial, business or other benefit, from 
the potential contract awarded to any of the competing, bidding-vendors listed above; and that, neither you nor 
members of your immediate family have any potential conflicts of interest in the organization(s) listed above, 
including any subcontractor referenced in their respective proposals, that could influence, or be reasonably 
perceived as influencing, your evaluation or recommendations for this solicitation. 

If it appears as potential conflict of interest between your official duties as an evaluator and your personal 
interest, you will be excused from participation by the Evaluation Committee Chair. Please return this form 
unsigned and a replacement evaluator will be assigned. You need not disclose the relationship or conflict. 

In addition the issue will be referred to the employee’s Department Director or Agency Head. Department 
Directors or the Heads of autonomous agencies shall be responsible for the preliminary examination and 
investigation of reports from employees of any violations which compromise the procurement process. If, 
following a preliminary examination and investigation, the Department Director or Agency Head finds evidence 
of a violation or finds that further investigation is warranted, a report shall be submitted to the respective Human 
Resources Office for potential disciplinary action. 

By signing below, I certify that I do not have, nor does any member of my immediate family have, 
any personal, financial, business, or other conflicts of interest in the bidding-vendors listed above. 


Signature 


Date 




Read to Achieve 2018 (RtAD) Evaluation Consensus Meeting notes 


Meeting Purpose 

Meeting to update team on the status of the RtA procurement 

Location 

Conference Room 385, Education Building, Raleigh. 

Date & Time 

March 8, 2019; 9:00 AM - 10:00 AM 

Facilitator(s) 

Tymica Dunn 

Next Meeting 

TBD M. 


Voting Member Participants 


Chloe Gossage 

Cynthia Dewey 

Kristi Day 

Lynne Loeser 

Matt Hoskins 

Pam Shue 

Susan Laney 

Thakur Karkee 









Non-Voting Member Participants 


Jonathan Sink 

Srirekha 

Viswanathan 

Tymica Dunn 









Agenda Items 

The agenda for this meeting was to update the evaluation team on the status of the procurement. 

Meeting Summary 

At the start of the meeting, the Procurement Officer informed the team that the participants will be addressed by 
the General Counsel. 


The General Counsel emphasized the importance of confidentiality and objectivity in an RFP procurement. He 
did add that one of the voting members breached the confidentiality of the procurement process which 
jeopardized the legality of this procurement. It should also be noted that the team did not reach a unanimous 
consensus on the choice of the finalist vendors. Because of these issues the current read to achieve 
procurement has to be cancelled again. Discussions were underway with DIT on the best possible approach to 
proceed. 

The meeting was adjourned. 


Next Steps: 




Guidance from 


Action items resulting from the meeting are as follows. 


Action lte>. 


Item 

Assignee 

Due Date 

Status 
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Read to Achieve 2018 (RtAD) Evaluation Consensus Meeting notes 


Meeting Purpose 

Consensus Meeting to recommend finalist for negotiations 

Location 

Conference Room 504 A, Education Building, Raleigh. 

Date & Time 

January 8, 2019; 1:30 PM - 3:00 PM 

Facilitator(s) 

Srirekha Viswanathan and Tymica Dunn 

Next Meeting 

TBD 


Voting Member Participants 


Abbey Whitford 

Chloe Gossage 

Cynthia Dewey 

Kristi Day 

Lynne Loeser 

Matt Hoskins 

Pam Shue 

Rebecca Belcastro 

Susan Laney 

Thakur Karkee 







Non-Voting Member Participants 


Mark Johnson 

Srirekha 

Viswanathan 

Tymica Dunn 









Agenda Items 

The agenda for this meeting was to recommend finalist for approval and negotiations. 


Meeting Summary 


The Superintendent thanked the evaluation team for their f 

He also mentioned that he had reviewed the prof: 
offerings. 



rd work and time spent on this most important RFP. 
the Holidays to get a full understanding of the various 


The Superintendent discussed his vision of empowering teachers and giving teachers their time back to teach. 
Empowering teachers include providing teachers the right tools; appropriate professional development and 
training. It is important to allow teachers to teach by reducing assessment time. 


He requested voting members to keep this vision in mind while making recommendations on the vendor(s) for 
negotiations. To maintain integrity of the process he stepped out and requested the voting team members to 
proceed with voting. 


The next stef#Tn this process i.e., recommendations by voting members, approval by Superintendent and 
negotiations were elaborated by the Business Owners and Procurement Officer. To further ensure that an 
impartial and unbiased process is followed, the voting members were provided ‘Post It’ cards to enter their 
recommendation^-, Sri tallied the votes and the recommendation was announced to the team. 



;ix (6) voting members recommended negotiating with Amplify only; 
hree (3) voting members recommended negotiating with Istation only; 

On^l) voting member recommended negotiating with both Amplify and Istation. 


The team discussed further and recommended that in order to align with the vision of the Superintendent, it is 
jortant that if negotiations are conducted with Amplify that the assessment measures are reduced to the core 
measures of DIBELS. The current implementation package includes TRC and it takes away significant teaching 
time. 
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Meeting Agenda & Summary 


The team also made a note that when negotiations are held with Istation it is important to further understand their 
recording and playback feature as it may also impact teaching time. 


In all the team felt that is important to understand the overall assessment time with both vendors and work 
towards reducing the assessment time. 


Next Steps: 

The Business Owner will provide an update to the Superintendent on the team’s recommend 
approval from Superintendent the next steps will be planned. 


Action items resulting from the meeting are as follows. 


ork 

“ 




Item 

Assignee 

Due Date 


Inform the State 

Superintendent of the team 
recommendation 

Dr. Pam Shue 

’- 9 - 19 

vY 

Gather negotiation questions 
and get team input on the 
questions 

Sri 

i 

Y 

Set up meetings with the 
finalist vendor 

Tymica Dunn 


) 






_ 
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Public Schools of North Carolina 


Read to Ac 




Business Owner(s): 

Dr Amy Jablonski 
Dr. Pamela Shue 

Project Manager: 

Srirekha Viswanathan 


Procurement Officer: 

Tymica Dunn 




Agenda 



BACKGROUND 




REVIEW STRENGTHS 

RANKING AND WEAKNESS 



NEXT STEPS 







Background 

SECTION 7.27 v o) T ht Q .ate Superintendent shall issue 
a Request for\ v o y osa l s (RFP) to vendors of diagnostic 
reading assessn, ; nt instruments to provide one or more 
valid, reliaJe, formative, and diagnostic reading 
assessment i^s+ ument or instruments for use pursuant 
to G.S 1150-1/4.11. 

/t a r\inir. ^m, the diagnostic reading assessment 
irk + urne it or instruments provided by the selected 
ven^’ shall meet all of the following criteria: 

a. Yield data that can be used with the Education Value- 
Added Assessment System (EVAAS). 

b. Demonstrate close alignment with student performance 
on State assessments 

c. Demonstrate high rates of predictability as to student 
performance on State assessments 

SECTION 7.27.(c) The State Superintendent shall form 
and supervise an Evaluation Panel to review the 
proposals received pursuant to the RFP issued in 
accordance with subsection (b) of this section. The 
Evaluation Panel shall be composed of persons 
employed within the Department of Public Instruction. 

By December 1,2018, the Evaluation Panel, with the 
approval of the State Superintendent, shall select one 
vendor to provide the assessment instrument or 
instruments for the 2019-2020 school year. 




Where we are... 
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Evaluation Ranking 


Evaluation Criteria 


Substantial Conformity 
to specification 

RtAD Desired 
Specifications 

Proof of Concept / 
Demonstration 

Vendor Cost Proposal 

Strength of References 

Vendor Financial 
Stability 

Overall Rank 




Amplify Education Curriculum Associates Istatioi 
Inc. 


4 

1 

4 




& 


,v 






1 

1 

1 


The evaluation criteria are stated in relative order of importance. 


NWEA 


2 

1 

1 


Public Schools of North Carolina 








Evaluation Ranking 

Ranked 4 th NWEA (MAP Assessment) 

Strengths 

1. The computer adaptive nature of the assessment helps each student to stay engager 

2. Good Reporting feature based on the data collected from screening measure. 

3. Parent communications could be available in multiple languages other tharyjpanisT 
Weakness 
1 . 

2 . 


3. 

4. 

5. 

6 . 


The Progress Monitoring tool currently under development is t 
assessments can only be given three times a year. 

This tool cannot accurately identify risk indicators for dyslex 
proposal was that the developers expect the service to be 
assessment. 


The vendor has mentioned that they wi 
SOC2 Type II audit. 


7. Equity of technology in schools ma 






nglish but that requires customizations. 


Progress monitoring is not yet in place and is currently under development. 

>nmProgi^s Monitoring tool that is going to exist because the benchmark 


company has not provided any data for the same. Their statement in the 
and specific screener for dyslexia. This will require multiple tools for 


Formative assessment is only given to some students biiause once the students read independently fluency assessment is optional. 

The proposal was not for a statewide implementation. 

negotiate with the state on the proposed security standards and has not given a clear timeline for the 


loss of instructional time. 


Public Schools of North Carolina 


Evaluation Ranking 


Ranked 3 rd Curriculum Associates (i-Ready Assessment) 

Strengths 

1. The computer adaptive nature of the assessments helps each student' 

2. The service has sound identification that is well described 

3. This service has the Standards Mastery Results (Student) re 
performed on an assessment, including how students perfor 

Weakness 

1. Fluency has not been developed and will not be ava 

2. i-Ready is not a reliable screener for dyslexia be' 
recognition. This will require multiple tools for ass 

3. The approach for service deployment is not slat 






another is a manual process and will take about 48 hours 


s teachers understand how students 
h skill in the assessment. 


2021 School Year. 

cks measures for fluency and non-sense word 
but by districts. Also student transfers from one district to 


Reporting feature requires a lot of customizatii 
will not be immediately available for thi 


s. Some reports have to be requested from the vendor and 
I Districts. 



5. Vendor has mentioned that the SOC 2 Type II assessment will be completed by Summer 2019. (This may 
compromise contract aware 1 ' 

6. Equity of technology in spools m% lead to loss of instructional time. 


w* Public Schools of North Carolina 



Evaluation Ranking 

Ranked 2 nd Istation (Istation Assessment) 

Strengths 

1. The assessment is adaptative in nature and adjusts to each student’s 

2. Teachers are incorporated in this service for early education,^ that 
teacher will playback and grade the student. 

3. In this assessment, students internalize the learning go 
student’s self-assessment process allows transition to jri 

4. Has robust reporting capabilities. 

Weakness 


4 > 


1 . 

2 . 

3. 

4. 





>ilities in early literacy, 
tiident’s reading is recorded and the 


e able to set the target for themselves. A 
nt learning. 


6 . 


Text fluency and oral language are not a part of the overall ability score. Fluency is a new assessment. 
No method to determine decoding. 

This assessment is not diagnostic in natu 
Istation is not a reliable screener fo 

like letter naming fluency. This will require%nTiltiple tools for assessmen 

Vendor has mentioned that the SOC 2 Type II assessment will be completed by Spring 2019. (This may 
compromise contract award) 

Equity of technology in schools may lead to loss of instructional time. 


because it lacks some key measures for dyslexia risk factors 
". it. 


Public Schools of North Carolina 



Evaluation Ranking 

Ranked 1 st Amplify Education Inc. (mClass Assessment) 

Strengths 

1. Offers online as well as observational assessment. 

2. The core measures of Dibels are a valid and reliable screenerfqj^isK factors for dyslexia. 

3. Offline assessment is available. 

4. The service has robust reporting capabilities. 

5. The service is SOC 2 Type II certified. 

Weakness 


1. There are many assessment measures that 




be turned off. 


mr Public Schools of North Carolina 



Cost Proposed 


Cost per student 


Amplify Education Inc. 

$25.78 

Curriculum Associates 

$22.48 

Istation 

$6.60 

NWEA 

$21.14 


Vendor 


Note: 

1. Costs will be negotiated with finalists. 

2. It includes potential cost for headsets. 

3. For Vendors who provided multiple costs. 




Sr Public Schools of North Carolina 



igher cost was considered. 








Next Steps 

1. Finalist Identification & Negotiations 

2. Best and Final Offer (BAFO) 




or 
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Read to Achieve 2018 (RtAD) Evaluation Consensus Meeting notes 


Meeting Purpose 

Consensus Meeting to rank the proposal vendors 

Location 

State Board Room, Education Building, Raleigh. 

Date & Time 

November 19, 2018; 8:30 AM - 5:00 PM & November 20, 2018 8:30 - 2:30 PM 

Facilitator(s) 

Linda Lowe and Srirekha Viswanathan 

Next Meeting 

TBD M. 


Voting Member Participants 


Abbey Whitford 

Amy Jablonski 

Chloe Gossage 

Cynthia Dewey 

Kristi Day 

Lynne Loeser 

Matt Hoskins 

Pam Shue 

Rebecca Belcastro 

Susan Laney 

Thakur Karkee 






Non-Voting Member Participants 


Courtney Moates 

Constance Bridges 

Deborah Wilkes 

Erika Berry 

Giancarlo Anselmo 

Gin Hodge 

Julien AlHour 

KC Hunt 

K.C. Elander 

fcLinda Lowe 

Meera Phaltankar 

Mia Johnson 

Paola Pilonieta 

Shaunda Cooper 

Srirekha 

Viswanathan 

Tonia Parrish 

Tymica Dunn 





Agenda Items 

The agenda for this meeting was to discuss the evaluation notes from the independent reviews by voting and non¬ 
voting members, reach consensus to rank the propose 3 ana determine the next steps in this procurement. 

This meeting summary includes notes from the meeting on 11-19-2018 and 11-20-2018. 

Meeting Summary 

( 11 - 19 - 2018 ) 

1. Sri kicked off the meeting by thanking the participants for their thorough review and participation at the 
consensus meeting. 

2. The intent of the meeting and the approach to evaluate all the criteria were discussed - including being 
objective, impartial, unbiased and fair in all aspects of the evaluation process and arrive at a consensus. 
All proposals should be ranked consistently. Consensus means general agreement and not unanimity. 

3. The six evaluation criteria in proposal were reiterated: 



Substantial Conformity to Solicitation Specifications 
RFP Desired Specification 
Proof of Concept/Demonstration 
ndor Cost Proposal 
endor Relevant Experience and Reference Checks 
Vendor Financial Stability 


responsive vendors were evaluated on all six evaluation criteria. 

evaluate substantial conformity to specifications the team unanimously agreed to take the following 
approach: 

a. Review the legislatively mandated specifications for all responsive vendors. 

b. Vendors who were deemed substantially conforming to statutory requirements to be further 
evaluated for all RFP specifications to ideally reach a group agreement and further rank the 
vendors. 
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Meeting Agenda & Summary 



c. Those vendors that were not substantially conforming to statutory requirements were ranked 
lower by the team for this evaluation criteria. 

6. The following ranking was used for each specification - 

a. “Yes” implies conforms to specifications. 

b. “No” implies does not conform to specifications. 

c. “MayBe” implies that the team is unsure about conformity. 

Discussion during the consensus meeting is summarized below. The voting members were issued three co 
cards - Green to show compliance to specification, Pink to show that the specification was not complied with 
Yellow to show Maybe there was compliance. In the case of Maybe responses, further clarifications may occur 
during negotiation prior to Best and Final Offer (BAFO) submission and Award. Negotiation questions matter for 
Vendors in the competitive range that are selected for further consideration. The voting members discussed each 
mandatory requirement in full and arrived at a consensus by showing the appropriate cards. Outcomes from 
consensus meeting for the various specifications are provided in a separate spreadsheet for each bidder with 
appropriate strengths and weaknesses. 

The proposals were taken up in an alphabetical order for ranking. 

1. Substantial Conformity to Specifications 

Review of Legislated Specifications 
Amplify Education Inc.: 

Business Specification 1: 

Describe how the proposed solution directly assesses read ig and pre-reading behaviors to support student’s 
learning development at the various grade levels to info^ istruction, including any observation-based practices 
if applicable: 

oral language (expressive and receptive 
phonological and phonemic awa 
phonics 
vocabulary 
fluency 

comprehension 




Consensus Ranking: The voting members were unanimous in their agreement that Amplify complied with this 
specification. Twcmf the voting members mentioned that while online versions are available for students with 
appropriate self-regulation and computer skills; teachers continue to have the option to directly assess/observe 
students. The voting members were 11-0 Yes on Amplify’s ability to comply with this specification. 


Business Specification 3: 

Describe the validity and reliability of the assessment in the following areas: 
oral language 

phonological and phonemic awareness 
inics 

d. vocabulary 

fluency 

f. comprehension 
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Meeting Agenda & Summary 


Consensus Ranking: 


Two of the voting members mentioned that the data is good and reliable on most assessments using DIBELS. 
Oral Language reliability data was sound. TRC data for Inter Rater Reliability (IRR) is low in many areas, 
voting members voted 10 Yes and 1 Maybe. 

Negotiation Question: TRC online shows concurrent validity to two measures of reading. Need the alph 
for the lower online TRC book levels. Early literacy measures to be included as part of negotiati 




ification because the team felt 
d areas that need intervention. 


ks are met by the proposed 


Business Specification 5: 

Describe how the assessment identifies and reports students who may need intervention and enric 


Consensus Ranking: 

The voting members were unanimous in their Yes votes for Amplify for this s 
that multiple reports and data are available for teachers about instruction 


Business Specification 6: 

Describe how the following characteristics for progress monitoring be 
solution: 

a. brief, 

b. repeatable, 

c. sensitive to improvement over time, including short 

d. multiple equivalent forms of screening assessm 

(weekly or every other week), 

e. 

f. 

g- 

h. 

i. 



the teacher to gauge short term growth 


J- 


reliable, 
valid, 

measure accuracy and fluency with skills 
quantitative results charted over time to calculate and document rates of improvement 
Allow for off-grade level progress monitoring 

Ability for the results to be graphed against a goal (national norms and/or individualized goals) with 12-14 


data points in 10 weeks’ time. 


Consensus Ranking: 

The team voted 8 Yes and 3 Maybe on the question of Amplify’s progress monitoring meeting the characteristics 
defined above. Some of the team members felt that TRC did not meet all of the above characteristics; however, 
the DA2yLas an outcome measure of reading comprehension does meet. The team felt that teachers choose 
their own book outside of the Atlas set for progress monitoring, which probably impacts reliability and validity. 

Question: As part of further negotiations, the team agreed that further data is required on TRC’s 
customizations needed for Oral Language measures. 

isiness Specification 8: 

Describe how the measures align with best practices and adequately and accurately identify indicators of risk for 
dyslexia in grades K-3 as outlined in NC Session Law 2017-127: 
n||6 ://www.ncleg.net/Sessions/2017/Bills/House/PDF/H149v4.pdf 
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Consensus Ranking: 

The team was unanimous on Amplify’s ability to meet this specification and voted a 11 -0 Yes on this specification. 
The SMEs in the area of dyslexia mentioned that Amplify’s tools are predictive of reading outcomes and include 
domains known to be impacted by dyslexia including phonological awareness and rapid naming. They also felt 
that DIBELS is sufficient as dyslexia screener although it does not get into higher level screening. The core 
measures of DIBELS have always been recognized as valid and reliable screener for risk factors for dyslex 
The only drawback of DIBELS is they do not get into the advanced levels of phonological awareness forj 
grade and beyond. 

Business Specification 9: 

Describe how the system uses developmentally appropriate practices to assess K-3 students. 

Consensus Ranking: 

The team voted 11 -0 Yes on Amplify’s ability to appropriately assess K-3 students. Receptive and expressive 
assessing options are available with this service. There was also the Observational and Online means of 
assessing students. 

Business Specification 10: 

Describe how the system incorporates educators and/or students u 
reading behaviors. 

Consensus Ranking: 

The team was unanimous in their votes about Amplify’s abil 
students and voted 11-0 Yes for this specification bee 
assessment. 


Business Specification 11: 

Describe how the proposed solution is 




assess reading and pre- 


ading and pre-reading behaviors of 
ity of online and observational 


ing assessment(s) tool for grades K, 1,2, 3. 


Consensus Ranking: 

The team voted 9 Yes and 2 Maybe on this specification. Some members felt that the proposal response did not 
adequately respond to this question. While Amplify shared research in support of formative assessment, the 
response did not include how the proposed solution is a formative reading assessment for grades K-3. Some 
SMEs also mentioned that individual skills measured by DIBELS assessments lend to formative assessment of 
different isolated skills. However, TRC component did not appear to be easy for formative reading assessment. 


Business Specification 12: 

Describe how the proposed solution is a diagnostic reading assessment(s) tool for grades K, 1, 2, 3. 

Consensus Ranking: 

The team voted 10 Yes and 1 Maybe. There was a question whether the TRC and MSV type analysis is truly 
diagnostioin nature and whether TRC’s diagnostics capacity is dependent on the teachers’ ability to interpret the 
student’s responses. 


C Z, 

Busine 


usiness Specification 15: 


Describe how the proposed solution minimizes impact to instructional time while ensuring formative and 
diagnostic assessments are conducted. Provide estimates of assessment time, for both benchmarking and 
progress monitoring, per student per grade. 
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Consensus Ranking: 

The team voted 7 Yes, 3 No and 1 Maybe on this specification. Some members felt that the response was 
described well, however the estimate of time was not answered. It was unclear as to the estimates of 
assessment time, for benchmarking and progress monitoring per student per grade level. Some voting memt 
pointed out that time differentials between the online and observational versions of assessment was provic 
the demonstration clarification document. It was also mentioned that TRC assessment could take longer and 
would require further negotiations and customizations. 

Business Specification 17: 

Describe how the content standards will be aligned and realigned to State Board of Education 
Standard Course of Study (Spring 2017). Provide specific mapping to the current standards. 
http://www.ncpublicschools.org/curriculum/languagearts/scos/. 



is also i 




Consensus Ranking: 

The team voted 7 Yes, 2 No and 2 Maybe for this specification. The members pointed out tli&t daring the demo 
the online instruction was aligned, however the gaming piece was not aligned to the Standard) Course of Study. 
There were insufficient examples for alignment for the assessment piec 
assessment questions are aligned to the ELA SCoS. 

Business Specification 19: 

Explain how the proposed solution can yield data that can 
information that explains any alignment or relationship bet' 

Assessment System (EVAAS). http://www.ncpublics 
https://www.sas.com/en_us/software/evaas.html 


Consensus Ranking: 

EVAAS expert in the evaluation panel mentioned 
unanimously voted 11 -0 Yes on Amplify’s ability to p 




to how the 


% 

fused with EVAAS. Describe and provide any 
en the assessment and the Education Value-Added 
1 g/effectiveness-model/evaas/ 


service can yield data required for EVAAS and the team 
data for EVAAS. 


Business Specification 24: 

Describe how the Benchmarking process occurs in the proposed solution. NCDPI expects benchmarking three 
times a year for grades K, ip 

Consensus Ranking: 

The team voted 11 -0 Yes on Amplify’s ability to Benchmark by State Board’s guidelines. 


Reporting Specification 3: 

Reporting feature is expected to provide the following capabilities: 
timely assessment results to teachers/administrators 
timely assessment results to parents/guardians 

reporting results at the district, school, grade, teacher, group, and individual student level by all 
roups ESSA 

end-of-year student summary report for cumulative folder 
historical data year after year to identify consistent gaps and learning trends for district, school, grade, teacher, 
up, and individual student level by all subgroups 
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For each of the above, provide a timeframe for how frequently the data is refreshed (real-time, on demand, or 
some other interval). 


Consensus Ranking: 

The Business Tearn voted 11 -0 in favor of Amplify’s service to provide timely assessment results. The team 
that the teacher reports are easy to read and interpret. Reporting feature provides drill down capability ir 
previous year’s assessment results. The service allows creating unique groups and assign view rights 1 
assorted individuals. 


Reporting Specification 4 

Provide communication to parents in a format that is clear and easy to understand after each 

Consensus Ranking: 

The team voted 7 Yes and 4 Maybe for this specification. The team felt that there shoulcn^differentlnethods of 


team felt 



■ent me 
ommur 


communicating to the parents and this is not explained clearly. Some members felt that parent communication 
should be in different languages and this was not clear. The team agreed that the Home Connect letters are 
easily understandable. 

Negotiation Question - Need for engagement webinars be archiv 


iarents. 


Technical Specification 6 

This service will be classified as “Program Critical/Moderate” based on the sensitivity of data used, the security 
controls under the “Moderate” category column need to be implemented. The vendor’s security policy should 
include all the control categories as specified under “Moderate” classification. Please refer to pages 4 through 10 
for the security control baselines table in the State Information Security Manual document. For example, AC-1 
(Access Control Policy and Procedures) under “Access Control” Family/Category is discussed in detail in the 
NIST 800-53 document. 

NC Statewide Information Security Ma nual - 
https://it.nc.gov/documents/statewide-information-security-manual 

a) Describe how you will ensure compliance to the NC Statewide Security Manual. 

Consensus Meeting - Without much detail the vendor said they would comply with the Security Manual. The 
use of developers in Ukrainsby this service was brought up. There is a letter from the vendor about them not 
using foreign workers for development under the current contract. This needs to be further clarified as to whether 
the request is for new RFP or is th%ifendor currently engaging their offshore developers for development? This 
will be a risk to be escalated if vendor currently engages this development team. 

The Securit^KME did clarify that the intent of the question is whether data goes to Ukraine and whether the 
developers from Ukraine can log in and see the data. The vendor has mentioned that the production data will not 
be shared. Previously under the current agreement, DPI verified that all data resides within the United States. 
Clarification was also provided that under the current contract with DPI the vendor has demonstrated compliance 
with the NC Statewide Security Manual. If they had not, DPI would not have been approved to renew contracts by 

team voted 11 Maybe because clarification needs to be sought from the vendor about use of developers in 



gotiation Question - To clarify if the request for offshore development is for new RFP or is the vendor 
ently engaging their offshore developers for development? This will be a risk to be escalated if vendor 
currently engages this development team. 
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Technical Specification 35 

Provide a 3rd party attestation, one of the following based on the system proposed: 

SOC2 Type II, SSAE-16, FEDRAMP, ISO 27001 

Consensus Meeting 

The team voted 11 Yes to this question because they were informed by IT Security expert in the tean^ th 
vendor has completed the SOC 2 Type II audit. 


Project Management Specification 1 

Include an initial schedule and the associated Work Breakdown Structure (WBS) for the propose: 
plan. The Project Schedule in the proposal to include significant phases, activities, tasks, milestone: 
requirements necessary for NCDPI to evaluate the plan. 


Consensus Meeting 

The voting team unanimously voted Yes for this question because this 
School Year. The team agreed that Project Schedule is well laid out 
under the new contract then it can be completed prior to the start of: 
added in the project plan is the timeframe for disaster recovery testir 



Negotiation Question - Update schedule to include disaster recovery tes 




e can be implemented for the 2019 
further enhancements are required 
School year. One thing that needs to be 


The team moved on to review Curriculum Associates for m 

Curriculum Associates (i-Ready) 

Business Specification 1: 

Describe how the proposed solution directly assesses reading and pre-reading behaviors to support student’s 
learning development at the various grade levels to inform instruction, including any observation-based practices 
if applicable: 

a. oral language (expressive and receptive) 

b. phonological and phonemic awareness 

c. phonics 

d. vocabulary 

e. fluency 

f. comprehensjfi 



Consensus Rant 

The voting member3Fybted 8 Nos and 3 Maybe. Some voting members felt that the vendor is currently not 
measuring oral language and fluency. The vendor’s rationale is that teacher interaction needed for these 
leasures. Also, in the demonstration clarifications this vendor has indicated that the "We propose working with 
NCDPI to identify one of the current traditional teacher-administered fluency assessments from another vendor 
^_j^eit available to all RTAD participants, at no additional cost to our original RTAD RFP response." which 
raised many questions. Some Subject Matter experts mentioned that just measuring comprehension skills without 
fluency raised the question of how the foundational skills were assessed. They had questions around how a 
student could be good on foundational reading but just will not be a fluent reader. Some members felt that the 
observational aspect is missing. In early reading assessment it is important to ask students to read aloud and 
that key component is missing. The team felt that the company did not understand the value of adding fluency 
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and that it was added because of demand and cited that timeline needs to be adjusted and planned to onboard 
another vendor, if this vendor is chosen as the finalist. The team also was concerned about having two different 
normative data sets one for fluency and another for all other measures. The measures for oral language were 
also very limited. CA has an estimated timeline for fluency development and does not say how close they are to 
complete development. 

Negotiation Question: Need to discuss how the vendor would add the fluency component with any co 
implications. More details on how they can present to the end user as a one vendor package are neede 
firmed-up timeline for fluency development. 


Business Specification 3: 

Describe the validity and reliability of the assessment in the following areas: 

a. oral language 

b. phonological and phonemic awareness 

c. phonics 

d. vocabulary 

e. fluency 

f. comprehension 


Consensus Ranking: 

The team voted 10 No and 1 Yes for this requirement bee 
mentioned that some team members were not able ti 


Business Specification 5: 

Describe how the assessment identifies and re 


Consensus Ranking: 

The team voted 1 Yes 4 No and 6 Maybe, 
other voting members agreed to that. The 
and that is a concern to identify students i 





d 

w*. w . .-3u. Get ci 

X/ 


ons cited above for fluency. It was 
psychometric data. 


o may need intervention and enrichment. 


said that the enrichment was not clear in their proposal and the 
was doubtful about the validity and reliability of the assessment 
ed appropriately. 


Business Specification 6: 

Describe how the following characteristics for progress monitoring between benchmarks are met by the proposed 
solution: 

brie^ 
repeatable, 

sensitive to improvement over time, including short term change 

iple equivalent forms of screening assessments that enable the teacher to gauge short term growth 
very othCTweek), 

liable, 

sasure accuracy and fluency with skills 

quantitative results charted over time to calculate and document rates of improvement 
Allow for off-grade level progress monitoring 

Ability for the results to be graphed against a goal (national norms and/or individualized goals) with 12-14 
data points in 10 weeks’ time. 
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Consensus Ranking: 

The team voted 10 No and 1 Maybe on this question because of the reliability and validity of the assessment. 
Progress Monitoring appeared to be a shortened version of the assessment and the team mentioned that to be a 
concern. Reliability almost always goes down to some extent when you reduce test items. Progress Monitorir 
itself takes 15 minutes. 

Business Specification 8: 

Describe how the measures align with best practices and adequately and accurately identify indi 
dyslexia in grades K-3 as outlined in NC Session Law 2017-127: 
http://www.ncleg.net/Sessions/2017/Bills/House/PDF/H149v4.pdf 

Consensus Ranking: The team voted 11 Nos because they did not think this servic 
for dyslexia because it lacks measures for fluency and nonsense (or pseudoword) won 


Business Specification 9: 

Describe how the system uses developmentally appropriate practices to 


Consensus Ranking: 

The team voted 1 Yes 6 No and 4 Maybe. It was mentioned that it is to 
learners and particularly for economically disadvantaged chij^n^ho 
children would have to master computer skills before they can be assessed. 





this is a problem for early 
ccess to computers. Young 


ing digital devices to assess reading and pre- 


Business Specification 10: 

Describe how the system incorporates educat oxs a nd/or 
reading behaviors. 

Consensus Ranking: 

The team voted 1 Yes 1 No and 9 Maybe. Some team members felt that prereading behaviors cannot be 
assessed completely online. A voting member mentioned that online assessment can be limited to identification of 
sound but was concerned about how production of sound can be assessed for young children without the fluency 
component in place. The service has sound identification that is well described. 


Business Specification 11: 


olution is a 


Describe how the proposed sol 

Consensus 'anking: 

The team voted 9 Yes, 1 No and 1 Maybe. 


formative reading assessment(s) tool for grades K, 1, 2, 3. 


Business Specification 12: 

scribe how the proposed solution is a diagnostic reading assessment(s) tool for grades K, 1,2, 3. 

us Ranking: 

voted 9 Yes, 2 Maybe. Some team members cited that in the proposal response the vendor had noted 
following "The Standards Mastery Results (Student) report helps teachers understand how students 
ormed on an assessment, including how students performed on each skill in the assessment. This report also 
displays the actual assessment taken by a student along with the correct answer, the student’s answer, and any 
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misconceptions that may have led to an incorrect or partially correct answer. This information can help teachers 
understand which concepts an individual student is struggling with and potential reasons why." 


Business Specification 15: 

Describe how the proposed solution minimizes impact to instructional time while ensuring formative and 
diagnostic assessments are conducted. Provide estimates of assessment time, for both benchmarking £ 
progress monitoring, per student per grade. 



Consensus Ranking: 

The team voted 1 Yes, 10 Maybe. Lynne mentioned that the assessments take 48 minutes and one membe 
mentioned that this company mentioned that the child gets to take 1 day or 2 days to complete the assessment 
and questioned the reliability and validity of the test results on the whole. Should we need some guidelines 
around how long assessments can take by grade level should this vendor be a finalized? Some team members 
mentioned that the end user will have to make some movement by moving the mouse. The technology 
representatives mentioned that the session appears to remain active upon clicking on the web pagj^This needs 
to be clarified. 


Negotiation clarification: Is the activity on the screen by clicking or by mouse m 


Business Specification 17: 

Describe how the content standards will be aligned and reali 
Standard Course of Study (Spring 2017). Provide specific 
http://www.ncpublicschools.org/curriculum/languagearts/s< 



ducation adopted ELA 
standards. 


Consensus Ranking: 

Some voting members mentioned that the continuum is not^igned to standards and it was a forced alignment. 
The standards mastery examples presented were not aligned and the team voted 3 Yes, 6 No and 2 Maybe. 


Business Specification 19: 

Explain how the proposed solution can yield data that can be used with EVAAS. Describe and provide any 
information that explains any alignment or relationship between the assessment and the Education Value-Added 
Assessment System (EVAAS). http://www.ncpublicschools.org/effectiveness-model/evaas/ 

https://www.sas.com/en_us/software/evaas.html 




/ 


Consensus Ranking: 

EVAAS expert in the evaluation panel mentioned that the service can yield data required for EVAAS and the team 
unanimously voted 1 Tj-0 Yes on Curriculum Associates’ ability to provide data for EVAAS. 


Business Specification 24: 

Describe how the Benchmarking process occurs in the proposed solution. NCDPI expects benchmarking three 
times a year for grades K, 1,2 and 3. 

sensus Ranking: 

nanimously voted Yes for this specification. 



sporting Specification 3: 

'porting feature is expected to provide the following capabilities: 
a. timely assessment results to teachers/administrators 
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b. timely assessment results to parents/guardians 

c. reporting results at the district, school, grade, teacher, group, and individual student level by all 
subgroups ESSA 

d. an end-of-year student summary report for cumulative folder 


historical data year after year to identify consistent gaps and learning trends for district, school, grade, teacl} 
group, and individual student level by all subgroups 

For each of the above, provide a timeframe for how frequently the data is refreshed (real-time, on 
some other interval). 


Consensus Ranking: 

The team voted 1 Yes, 9 No, 1 Maybe. The team was concerned about the complexity and difficu 
exporting to create subgroups. The wanted to know how long it would take to customize and add ad' 
report, should an outside request be made to get certain reports that are not readily 
the response time. 




hat will be 


e and cost impact 


after each benchmark. 


Negotiation Question: Can reports be customized for subgroups, 
if requests for custom reports are made? Should this be included in 

Reporting Specification 4 

Provide communication to parents in a format that is clear and easy to 

■ 

Consensus Ranking: 

The team voted 4 Yes, 2 No and 5 Maybe. Two team members said that they read in the response that the 
parent had to logon to get the report. It was clarified that the report is acceptable and that it can be printed and 
sent home by the teacher if the parent is unable to access a computer. 



Technical Specification 6 

This service will be classified as “Program Critical/Moderate” based on the sensitivity of data used, the security 
controls under the “Moderate” category column need to be implemented. The vendor’s security policy should 
include all the control categories as specified under “Moderate” classification. Please refer to pages 4 through 10 
for the security control baselines table inthe State Information Security Manual document. For example, AC-1 
(Access Control Policy and Procedures) under “Access Control” Family/Category is discussed in detail in the 
NIST 800-53 document. 

NC Statewide Information Security Manual - 
https://it.nc.gov/documents/statewide-information-security-manual 



a) Describe hokyo^ill ensure compliance to the NC Statewide Security Manual. 

Consensus Meetir 

The team voted a unanimous 11 Maybe for this specification as CA stated in their proposal response - "i-Ready 
gnostic is a SaaS product, so that some of the policies in the NC Statewide Security Manual do not apply, 
i/eveqwe feel i-Ready meets the intent of the security practices and policies as outlined.” In their subsequent 
^^^^Jon the vendor indicated that they will complete the SOC 2 Type II in the summer of 2019. The Security 
/Officer clarified that he cannot clearly say if they would want to negotiate because they were a SaaS shop. 



^gotiation Question - If this vendor is a finalist then their timeframe for SOC 2 Type 
also confirmation required about their compliance to the Security Manual. 


should be clarified and 
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Technical Specification 35 

Provide a 3rd party attestation, one of the following based on the system proposed: 

SOC2 Type II, SSAE-16, FEDRAMP, ISO 27001 

Consensus Meeting 

The team voted 11 Maybe for this specification because of the reasons cited above for Technical Specificatipn 6. 
Currently DIT will not permit agencies to issue new contracts for moderate level solutions that have not been 
received an acceptable 3 rd party attestation. 

Project Management Specification 1 

Include an initial schedule and the associated Work Breakdown Structure (WBS) for tte proposed implementation 
plan. The Project Schedule in the proposal to include significant phases, activities, tasks, milestones and resource 
requirements necessary for NCDPI to evaluate the plan. ^ 

Consensus Meeting 

It was mentioned that the approach defined is not statewide and can end up very time consuming. Plus, the fact 
that fluency component needs to be added on requires planning and additional time. Team has to factor other 
customizations for a statewide deployment for the 2019 School Year. Implementation of EdFi ODS is targeted for 
2020 in the proposal response. The team voted 10 No and 1 Maybe for this specification. 

The team moved on to reviewing Istation for mandatory sp«ifications. 

d pre-reading behaviors to support student’s 



itorm instruction, including any observation-based practices 


Consensus Ranking: The voting members voted 1 No and 10 Maybe for this specification. Some team 
memberewere concerned about the oral language and phonological awareness in this service. They were not 
sure ifihe measure for fluency was the correct measure. Additional equipment will be needed. The observational 
piece is optional. Some voting members felt that this was a low-level assessment especially for oral language. 


mentioned even though the proposal mentioned that the reading can be recorded for the teacher to 
and grade fluency, it was not demonstrated. Students speak into a microphone and there is nothing that 



ips them and that could be frustrating. 

W 

^btiEjjpn Question: Oral language assessment is unclear or missing and needs to be further understood. 

isiness Specification 3: 
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Describe the validity and reliability of the assessment in the following areas: 

a. oral language 

b. phonological and phonemic awareness 

c. phonics 

d. vocabulary 

e. fluency 

f. comprehension 

Consensus Ranking: 

The team voted 9 Yes, 1 No and 1 Maybe. Some SMEs mentioned there is a need for clear description of 1 
normed group demographics. Some team members also mentioned that each of theirmssessment measures did 
not check validity and reliability and that the sample size is very low. Their criterion measure was based on one 
school in Texas. The Subject Matter Experts felt that the AUC data was very low, and the sample size was for 25 
students. It was pointed out that there was a lot of their comparison to DIBELj; 

Negotiation Question: Can the language around composite score be 


Business Specification 5: 

Describe how the assessment identifies and reports students 



Consensus Ranking: 

The voting members were unanimous in their Yes vo 
piece was not clear. 






ntion and enrichment. 


It was mentioned that the enrichment 


onitoring between benchmarks are met by the proposed 


luding short term change 
ing assessments that enable the teacher to gauge short term growth 


Business Specification 6: 

Describe how the following characters 
solution: 

a. brief, 

b. repeatable, 

c. sensitive to improvi 

d. multiple equivalent 
(weekly or every other week 

reliable, 
valid, 

measure accuracy and fluency with skills 

antitative results charted over time to calculate and document rates of improvement 
for off-grade level progress monitoring 

for the/esults to be graphed against a goal (national norms and/or individualized goals) with 12-14 
in 10 weeks’ time. 

sus Ranking: 

oted 1 Yes, 5 No and 5 Maybe. It was mentioned that the company recommended monthly progress 
onitoring, but the service allows you to do as many as you want. 

Business Specification 8: 
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Describe how the measures align with best practices and adequately and accurately identify indicators of risk for 
dyslexia in grades K-3 as outlined in NC Session Law 2017-127: 
http://www.ncleg.net/Sessions/2017/Bills/House/PDF/H149v4.pdf 


Consensus Ranking: 

The team voted 9 No and 2 Maybe for this requirement. It was mentioned that the service was missing 
the key measures for dyslexia risk factor identification like letter naming fluency. 

Business Specification 9: 

Describe how the system uses developmentally appropriate practices to assess K-3 students 

Consensus Ranking: 

The team voted 7 No and 4 Maybe for this one. Some voting members were concern 
of this assessment and how appropriate it is for Kindergarteners and struggling learners, 
also for 40 minutes. 




e piece 
ent was 


ssess reading and pre- 


Business Specification 10: 

Describe how the system incorporates educators and/or students us 
reading behaviors. 


Consensus Ranking: 

The team voted 3 Yes, 2 No and 6 Maybe. It was indicated that online assessment for prereading behavior will 
be a concern given the fact the targeted audience is Kindergarteners. The team also mentioned that the vendor 
described better on incorporating teachers and it made a difference frofp the other online assessments. Teachers 
can also go back and listen to recording. 


Business Specification 11: 

Describe how the proposed solution is 



ing assessment(s) tool for grades K, 1,2, 3. 


Consensus Ranking: 

The team voted 7 Yes and 4 Maybe on this specification. It was mentioned that the strength of this service as 
compared to the other online assessment service in that from NCDPI's definition of formative assessment (pg. 9 
on RFP), students internalize the learning goals and become able to see the target themselves. A student’s self- 
assessment process marks the transition to independent learning. It was added that the evaluation team would 
like to see how this is driven by the solution and less by the teacher. 


Negotiation Cuestiqfi: Clarify how formative reading assessment is driven less by teachers and more by the 
solution. 

Business Specification 12: 

Describe how the proposed solution is a diagnostic reading assessment(s) tool for grades K, 1,2, 3. 


lsensus Ranking: 

feted 2 Yes, 2 No and 7 Maybe. It was mentioned that the service allows teachers to be diagnosticians 
and does not say much about how the service diagnoses reading deficiencies. It was felt that the nature of the 
ssessment is not diagnostic. 


Business Specification 15: 
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Describe how the proposed solution minimizes impact to instructional time while ensuring formative and 
diagnostic assessments are conducted. Provide estimates of assessment time, for both benchmarking and 
progress monitoring, per student per grade. 



Consensus Ranking: 

The team voted 3 Yes, 4 No and 4 Maybe on this specification. The team was not certain about where i 
instructional time was clearly addressed in the proposal response. One SME mentioned that the timefr 
addressed in question number 11 in the clarification document. The team was allowed to review the |<esp' 
again. Based on the review of the response the team came up with the votes. It was also noted that the 
assessment time far outweighs the impact to instructional time. 

Business Specification 17: 

Describe how the content standards will be aligned and realigned to State Board of Education 
Standard Course of Study (Spring 2017). Provide specific mapping to the current standards. 
http://www.ncpublicschools.org/curriculum/languagearts/scos/. 


Consensus Ranking: 

The team voted 3 No, 8 Maybe for this specification. It was pointed 
are not aligned to the current SCoS. The continuum that is provide 
NCSCoS and the questions/examples. 


Business Specification 19: 

Explain how the proposed solution can yield data that can be used with EVAAS. Describe and provide any 
information that explains any alignment or relationship between the assessment and the Education Value-Added 
Assessment System (EVAAS). http://www.ncpublicschools.org/effectiveness-model/evaas/ 
https://www.sas.com/en_us/software/evaas.htr 




tions and activities 
nment between 



Consensus Ranking: 

EVAAS expert in the evaluation panel mentioned that the service can yield data required for EVAAS and the team 
unanimously voted 11 -0 Yes on Istation’)^ abiliMltejfiyiae data for EVAAS. 


Business Specification 24: 

Describe how the Benchmaifcing process occurs in the proposed solution. NCDPI expects benchmarking three 
times a year for grades K, 1*2 and 3. 


Consensus Ranking: 

The team voted 10 Yes and 1 Maybe. One team member had questions around how the benchmark timeframes 
can be opened and closed. 

ues^ui How can benchmarking window be opened and closed. 

porting Specification 3: 

Sorting feature is expected to provide the following capabilities: 

5ly assessment results to teachers/administrators 
timely assessment results to parents/guardians 

reporting results at the district, school, grade, teacher, group, and individual student level by all 
ogroups ESSA 

d. an end-of-year student summary report for cumulative folder 
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historical data year after year to identify consistent gaps and learning trends for district, school, grade, teacher, 
group, and individual student level by all subgroups 


For each of the above, provide a timeframe for how frequently the data is refreshed (real-time, on demand, or 
some other interval). 

Consensus Ranking: 

The voting members voted 11-0 in favor of Istation's capability to provide timely assessment result: 
that the teacher reports are easy to read and interpret. 



Reporting Specification 4 

Provide communication to parents in a format that is clear and easy to understand after each bend 

Consensus Ranking: 

The team voted 3 Yes; 2 No and 6 Maybe for this specification. It was noted that there was no separate report for 
parents. Only a summary report for teachers was available for parents. Thare was a letter that could go home but 
it was not automatically system generated but had to be filled in by the teachers. It was felt that most reports for 
teachers cannot be interpreted by parents. 




Technical Specification 6 

This service will be classified as “Program Critical/Moderate” based on the sensitivity of data used, the security 
controls under the “Moderate” category column need to be implemented. Th^vendor’s security policy should 
include all the control categories as specified under “Moderate” classification. Please refer to pages 4 through 10 
for the security control baselines table in the State Information Security Manual document. For example, AC-1 
(Access Control Policy and Procedures) under “Access Control” Family/Category is discussed in detail in the 
NIST 800-53 document. 

NC Statewide Information Security Manual - 
https://it.nc.gov/documents/statewide-information-security-manual 


a) Describe how you will ensure compliance to the NC Statewide Security Manual. 

Consensus Meeting - The team voted 11 Yes because the vendor mentioned that they will agree to work with 
the state’s security manual. 

Technical Specification 35 

Provide a 3rd party attestation, one of the following based on the system proposed: 

SOC2 Type II, SSAE-ll, FEDRAMP, ISO 27001 

Consensus Meeting 

The team voted 11 Maybe to this question because the vendor has agreed to complete the SOC 2 Type II by 
ring 2019 in order to meet this requirement. Currently DIT will not permit agencies to issue new contracts for 
derate level solutions that have not been received an acceptable 3 rd party attestation. Obtaining approval from 
to award a contract with a vendor that does not already meet this requirement will be challenging. 

^Negotiation Question: Clarify and get the timeframe for SOC 2 Type II completion. 


Project Management Specification 1 
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Include an initial schedule and the associated Work Breakdown Structure (WBS) for the proposed implementation 
plan. The Project Schedule in the proposal to include significant phases, activities, tasks, milestones and resource 
requirements necessary for NCDPI to evaluate the plan. 

Consensus Meeting 

Their plan had training in mid and late August this will be too late for year-round schools. Data Integration (IAM/^ 
EVAAS) was not indicated in the Project Schedule. The team voted 2 Yes and 9 Maybe. 

Negotiation Question - Training dates need to be negotiated. Need a firm GoLive date. 

~ v 

Business Specification 1: 

Describe how the proposed solution directly assesses reading and pre-reading behaviors to support student’s 
learning development at the various grade levels to inform instruction, inclyding any observation-based practices 
if applicable: 

a. oral language (expressive and receptive) 

b. phonological and phonemic awareness 

c. phonics 

d. vocabulary 

e. fluency 

f. comprehension M 

Consensus Ranking: The voting members voted 7 No and 4 Maybe for this question for NWEA because Some 
team members pointed out that there were many assessment components and it was hard to sort out. There was 
MAP Growth, skills checklist etc. The servi ce did nc^directly assess oral language. It was brought to the team’s 
attention that on page 45, "Beginning in the 2019-2020 school year, we anticipate MAP Reading Fluency will 
include progress monitoring forms that can be used in between benchmark tests." While the benchmark system 
meets the requirements, the Progress Monitoring is yet to be in place. MAP was also planning to include audio for 
their K-1 class. 

Business Specification 3: 

Describe the validity and reliability of the assessment in the following areas: 


oral language 


a. 


I 


id phonemic awareness 




Ranking: 


The team voted 1 Yes, 6 No and 4 Maybe for this specification. It was mentioned that NWEA has fluency and 
comprehension threshold in MAP Growth Assessment. This will be problematic. 
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Business Specification 5: 

Describe how the assessment identifies and reports students who may need intervention and enrichment. 


Consensus Ranking: 

The voting members voted 4 Yes, 3 No and 4 Maybe for this specification. One team member mentioned the 
because she was not confident with the validity and reliability, she was not sure if the system can identify stu 
who need intervention. 

Business Specification 6: 

Describe how the following characteristics for progress monitoring between benchmarks are 
solution: 

a. brief, 

b. repeatable, 

c. sensitive to improvement over time, including short term change 

d. multiple equivalent forms of screening assessments that enable the ti 

(weekly or every other week), 

e. 

f. 

g- 

h. 

i. 


term growth 


J- 


reliable, 
valid, 

measure accuracy and fluency with skills 
quantitative results charted over time to calculate an 
Allow for off-grade level progress monitoring 
Ability for the results to be graphed against a goal 


data points in 10 weeks’ time. 



provement 


and/or individualized goals) with 12-14 


Consensus Ranking: 

The team voted 10 No and 1 Maybe. It was mehtioned that Progress Monitoring is under development. An 
evaluation team member mentioned that the under-development Progress Monitoring tool is the only Progress 
Monitoring tool that is going to exist because the regular assessment can only be given three times a year. The 
Progress Monitoring fix is their skills checklist. From a growth perspective going from a norm reference to a 
criterion reference and basing it on growth checklist can be problematic. 


Business Specification 8: 

Describe how the measures align with best practices and adequately and accurately identify indicators of risk for 
dyslexia in grades K-3 as outlined in NC Session Law 2017-127: 
http://www.ncleg.net/Sessions/2017/Bills/House/PDF/H149v4.pdf 


Consensus Ranking: 

The team voted all No on this specification. It was mentioned that their response was very brief and even their 
clarification was brief. The team was concerned about the statement in the proposal that the developers expect 
that the service is sensitive and specific to screening for dyslexia but no data is currently available. 

iness Specification 9: 

how the system uses developmental^ appropriate practices to assess K-3 students. 



onsensus Ranking: 
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The team voted 2 Yes; 3 No; 6 Maybe on NWEA's ability to appropriately assess K-3 students. It was mentioned 
that they had drag and drop at the demo that is not developmentally appropriate for the target group especially 
the kindergarteners. 


Business Specification 10: 

Describe how the system incorporates educators and/or students using digital devices to assess reading 
reading behaviors. 

Consensus Ranking: 

The team voted 3 Yes; 2 No; 6 Maybe. It was noted that there is no observational aspect to a: 
behaviors. 

Business Specification 11: 

Describe how the proposed solution is a formative reading assessment(s) tool for grade 

Consensus Ranking: 

The team voted 6 Yes and 5 Maybe on this specification. It was pointed out that from page 51 of the response to 
RPF, "NWEA recommends administering MAP Growth and MAP Reading Fluency formative assessments at 
regular benchmark intervals across the year in grades K-3. Once students can read independently with adequate 
rate, accuracy, and literal comprehension, MAP Reading Fluenc^jio longer needs to be given." This definition 
seems to suggest that formative assessment is only for some students and is a benchmark assessment. 



Business Specification 12: 

Describe how the proposed solution is a diagnostic 


Consensus Ranking: 

The team voted 3 Yes; 3 No and 5 Maybe. 

Business Specification 15: 

Describe how the proposed solution mini 
diagnostic assessments are conducted, 
progress monitoring, per student per grade. 





:ssment(s) tool for grades K, 1,2, 3. 


impact to instructional time while ensuring formative and 
estimates of assessment time, for both benchmarking and 


Consensus Ranking: 

The team voted 2 Yes, 3 No and 6 Maybe on this specification. This product does not have Progress Monitoring 
built yet. It can take up to an hour to complete benchmark and for students who are falling off, the teacher will 
have to go back and review the recording (total of 2 hours per student). This time needs to be added to the overall 
time. Thiiwendor talked about going to a lab and it depends on the school to have that kind of lab. A field member 
noted that network bandwidth limits the number of concurrent tests that a school can support in their labs in 
Buncombe county. Fewer than half of the students in a classroom can go online at the same time. So, in some 
districts, even when labs exist in schools with the proper computer equipment (with required high quality 
aerophones), students may not be able to be assessed at the same time due to noise levels and network 
limitations. 

It was also pointed out that even in those labs there may be a combination of iPads and Chromebook and the kids 
interact differently. This could present a problem for kindergartners because the teachers will have to train 
them accordingly. If it is a true formative assessment, then it does not take time away from instruction. Formative 
ssessments inform instruction and it is observational for younger kids. 


Business Specification 17: 
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Describe how the content standards will be aligned and realigned to State Board of Education adopted ELA 
Standard Course of Study (Spring 2017). Provide specific mapping to the current standards. 
http://www.ncpublicschools.org/curriculum/languagearts/scos/. 


Consensus Ranking: 

The team voted 6 No and 5 Maybe for this specification. It was noted that the questions in the examples 
aligned to Standard Course of Study and the chart was confusing. 

Business Specification 19: 

Explain how the proposed solution can yield data that can be used with EVAAS. Describe an 
information that explains any alignment or relationship between the assessment and the Educa 
Assessment System (EVAAS). http://www.ncpublicschools.org/effectiveness-model/evaas/ 
https://www.sas.com/en_us/software/evaas.html 


Consensus Ranking: 

The EVAAS expert in the team mentioned that the service can yield data 
unanimously voted 11 -0 Yes on NWEA's ability to provide data for EVA 

Business Specification 24: 

Describe how the Benchmarking process occurs in the proposed soluti 
times a year for grades K, 1,2 and 3. 

Consensus Ranking: 

The team voted 11 -0 Yes on this specification and requi 


the team 


pects benchmarking three 


Negotiation Clarification: Need clarification on t 




n the benchmark testing window. 


esting timeframes. 


Reporting Specification 3: 

Reporting feature is expected to provide the following capabilities: 

a. timely assessment results to teachers/administrators 

b. timely assessment results to parenti/guardians 

c. reporting results at the district, schoof grade, teacher, group, and individual student level by all 
subgroups ESSA 

d. an end-of-vear student summary report for cumulative folder 

historical data year after year to identify consistent gaps and learning trends for district, school, grade, teacher, 
group, and individual student level by all subgroups 


For each of the above, provide a timeframe for how frequently the data is refreshed (real-time, on demand, or 
some other interval’ 



sensus Ranking: 

team voted 11 Yes for this specification. It was mentioned that subgroup reporting is unclear. 


Reporting Specification 4 

vide communication to parents in a format that is clear and easy to understand after each benchmark. 
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Consensus Ranking: 

The team voted 2 Yes; 2 No and 7 Maybe. Some members were not sure what they had for parents. It was 
clarified that Progress Monitoring reports will be shared with the parents which will be too hard for parents to 
comprehend. 


Technical Specification 6 

This service will be classified as “Program Critical/Moderate” based on the sensitivity of data used, the : 
controls under the “Moderate” category column need to be implemented. The vendor’s security polic 
include all the control categories as specified under “Moderate” classification. Please refer to pages 4 througf 
for the security control baselines table in the State Information Security Manual document. For example,'AC-1 
(Access Control Policy and Procedures) under “Access Control” Family/Category is discussec 
NIST 800-53 document. 

NC Statewide Information Security Manual - 
https://it.nc.gov/documents/statewide-information-security-manual 

a) Describe how you will ensure compliance to the NC Statewide Security M 

Consensus Meeting - 

The team voted 11 No because this vendor would like to preserve their opportunity to negotiate on all security 
related questions. The vendor was interested in discussing a different security standard (CIS) rather than NIST, 
which is the standard followed by the State of North Carolina. In addition, this vendror’s solution appears to permit 
teachers to see data for students who are not their own. ^ w 



Negotiation Question - Need to find out what they 

Technical Specification 35 

Provide a 3rd party attestation, one of the follow 
SOC2 Type II, SSAE-16, FEDRAMP, ISO 27001 

Consensus Meeting 

The team voted 11 No to this 
SOC 2 Type II. Currently Dj^will 
have not been received an 



all the security questions. 


system proposed: 



n because the vendor wanted to preserve the opportunity to negotiate for 
not permit agencies to issue new contracts for moderate level solutions that 
ceptable 3 rd parfy attestation. 


Project Management Specification 1 

Include an initial schedule and the associated Work Breakdown Structure (WBS) for the proposed implementation 
plan. The Project Schedule in the proposal to include significant phases, activities, tasks, milestones and resource 
requirements necessary for NCDPI to evaluate the plan. 


Concensus Meeting^ 

ie voting team unanimously voted No. The implementation plan was not statewide and were asking for a 
nary point of contact for each charter and district. Their GoLive was May 27 th in the plan and their Quality 
jrance and testing was after the GoLive. 



Negotiation Question - Need to revamp schedule for a statewide implementation. 
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After the initial round of deep dive review of the mandatory specifications, the evaluation team ranked the vendors 
for substantial conformity of mandatory requirements and selected the vendors in the competitive range for deep 
dive of all RFP specifications. 


For the substantial conformity of legislative specification, the vendors were ranked as follows 

1. Amplify Education Inc. 

2. Istation 

3. Curriculum Associates 

4. NWEA 

Summary: 


<$> 


The team deliberated and summarized that in the case of Curriculum Associates (i-Readv), their fluency 
measure is not ready and will not be available until the 2021 School Year. There was also concerns and 
skepticism about how the foundational questions were assessed just by measuring comprehension skills without 
fluency. The measures for oral language were also very limited. Progress Monitoring appeared to be a 
shortened version of the assessment and reliability almost always goes down to some extent when you reduce 
test items. This service cannot be a reliable screener for dyslexia because it does not have fluency and nonsense 
word recognition, a means to assess a student’s ability to apply letter/sound knowledges unknown words, a core 
deficit in students with dyslexia. The continuum is not aligned tq_gtanoards and the examples presented were a 
forced alignment. The standards mastery examples presented at the demo were not aligned. This service 
requires a lot of reporting customization. It also requires manual intervention for student transfers. The company 
plans to complete SOC 2 Type II audit in the summer of 2019. This presents a challenge when negotiating with 
DIT for contract award and will extend the contract award tirrkline. The implementation plan proposed is not for 
statewide implementation and appeared time consuming which brought to question the ability to deploy for the 
2019 School Year. 

In the case of NWEA, this service does not directly assess oraTlanguage. Progress Monitoring is not in place and 
implementing this solution will be a problem for the 2019 School year. Since this service does not have Progress 
Monitoring in place in the interim the growth checklist is used. This service does not screen for many of the key 
indicators of risk for dyslexia. This tool is also not a good diagnostic and formative assessment screener. Their 
example questions were not aligned to standards. Their project implementation timeline is not for a statewide 
implementation and there were serious doubts how the statewide implementation can be handled. There were 
concerns raised about teachers viewing other teacher’s students. Time has to be accounted for customizations to 
be FERPA compliant. The vendor has indicated negotiating state security standards and has not given a clear 
response for SOC 2 Type II audit requirement which will delay approval from DIT or even potentially get the 
contract award rejected. 

Considering all of the above for both the vendors, the team decided that they will not do a deep dive review of the 
non-mandatory questions for Curriculum Associates and NWEA. The team completed a deep dive of all 
substantial conrlrmity items for Amplify Education Inc and Istation. Their independent vote count was retained for 
the non-mandatory substantial conformity items for NWEA and Curriculum Associates. 

iei^ 'Jon-Mandatory substantial conformity specifications: 



oontinued with ranking substantial conformity for Amplify Education Inc and Istation. There were 
liscussrofls of both strengths and weaknesses for each vendor. The following is a summary of key discussion 
joints for each vendor followed by ranking. 
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Amplify Education Inc. 

Business Specification 

While reviewing how the solution adapts as students gain mastery and have demonstrated proficiency, it was 
pointed out that this solution is not adaptive and that there were serious doubts about how it can adapt as 
students gain mastery. There were questions about how grade level determines the universal screening 
student mastery of content. 

The team was concerned about the proposed training for Master Literacy Trainers. Also, the proposal res] 
did not adequately include strategies for ensuring consistent scoring to evaluate training effectiveness. This 
to be further negotiated with the vendor during negotiation. 

Reporting Specification 

Julien pointed out to the team that the reporting permissions need to be enhanced, 
satisfied with Amplify’s reporting capabilities and the reporting offering. 




was 


Technical Specification 

At the start of the evaluation of technical specifications with permission from Procurement, an evaluation team 
member expressed concerns about how the vendors support for iPad was not x adequate and that the districts had 
difficulty accessing the service with the newer iPads. The District representatives at the evaluation expressed 
that in their specific districts there were no real trouble accessing the service using the newer iPads. The only set 
up required was using the correct URL to get to the 1AM integrated service. Some teachers were still using the old 
URL for login. 

Concerns were also expressed with the ETL process with this vendor and said that this needs to be enhanced if 
this vendor is awarded the contract. 

On the question of 1AM integration, it was expressed that even though the current service is 1AM integrated, the 
protocols can be improved, and that there were concerns with the architecture. It was also pointed out that the 
offline access of the service required additional coding to remain compliant. The team voted a unanimous Maybe 
for the 1AM integration question based on this need to enhance. 

There were questions about tier 1 through tier 3 support and it was pointed out that the response time for closing 
tickets should be negotiated with the vendor should tnfe vendor be selected for further consideration and 
negotiations. 

This vendor has a SOC 2 Type 2 Audit completed by a third-party auditor and has also been completing 
penetration testing under the current contract and is highly rated in terms of security. 


There was a question about the physical audit of the data center by NCDPI. This question is irrelevant now 
because most SaaS service is hosted in the cloud. Although there was a unanimous No for this question, Sri and 
Tymica were advised to remove this question from the future RFPs. 


Project Management 

During discussions it was pointed out that the key Technical resumes were lacking. Maturity to manage technical 
aspects needs to be improved. Vendor failed to acknowledge agreement to this term: "Prior to making personnel 
changes for key human resources outlined in the project plan, the vendor must provide an opportunity for NCDPI 
review resumes and transition plan and request a meeting with replacement resources." This needs to be 
lified. 

team unanimously voted 11 Maybe for Project Management resumes. 

this weakness there were strengths in the implementation approach which is statewide supporting 
regional model. There were strengths in Project Processes as well. 
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Service Level Agreement 

Amplify had shared the current service level agreement that is in place. If this vendor is selected for negotiation 
the tiered support process, the associated reports and timeframe for issue resolution should be improved. 


Negotiation Question 

1. Professional Development and training needs to be enhanced and negotiated 

2. Enhancement of Reporting Permission 

3. Additional insights on the ETL process 

4. Tiered support to be finalized and response time for issue to be negotiated 

5. NCDPI to see the technical resumes and confirm their review and approval of resouro 



The team moved on to review Istation. 

Istation 

Business Specification 

The team had serious concerns about the validity and reliability of the service as a universal screener because 
the results presented in the RFP was from a study using ISIP Early Reading that was conducted in five 
elementary schools from a north Texas school district. Some voting members questioned the study parameters 
and its transferability to NCEOG standards. While main classification study had a very good ‘n’ it again 
demonstrates predictive validity from one district in Texas. Classification accuracy data in the RFP Attachment 1 
suggests very low sample sizes when determining AUC data.,#ie tpam also felt that the RFP response didn't 
specifically address sub-group and like-peer group reportingfeatures to assess progress. It would be hard to use 
the system for frequent monitoring needed for SLD policy compliance. Some of the voting team members were 
concerned with the reports noting effectiveness of com instruction. Group intervention effectiveness for 
supplemental support was also unclear in the proposal. 


The team liked the use of a consultative approach described in the proposal to designing PD based on local 
needs. There were also strengths in the virtual modules offered. However, there should be more specifics for 
each content strand of PD offered. This needs to be clarified during negotiations. 


is nee 


Reporting Specification 

Reporting appeared to be ea 
needs to be further negotiati 


) use but the 
is on the SLA 







ire was limited specifics on the report to track service usage. There 
iports. 


Technical Specification 

It was clarified to the team that the architecture approach for this service was current. There needs to be 
clarification omthe penetration testing and frequency. 

With regards to service scaling it was pointed out that this service is not elastic, and it needs to be negotiated with 
the vendor during negotiations. The solution maps to CEDs but there still need to be negotiations in the use of 
SIF. The data transfer capabilities are not quite state of the art. Otherwise the TASD shows well. 


There was a question about the physical audit of the data center by NCDPI. This question is irrelevant now 
because most SaaS service is hosted in the cloud. Although there was a unanimous “No” vote for this question, 
Sri qnd Tymica were advised to remove this question from the future RFPs. 

J roject Management Specifications 

flnere were serious concerns about the proposed Project Manager. The resume shows training. However actual 
Cementation experience is missing. The team agreed that negotiation Clarifications should be undertaken to 
probe deeper to see how the essential project management and technical roles will be staffed to ensure success. 
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The team also agreed that clarification should be sought on how User Acceptance Testing (UAT) bug fixes will be 
conducted and should be incorporated in the plan. 


SLA Specifications 

There were questions around SLA availability and how the vendor reported that the 99.9% will be ensurec 
non-peak time. The team had questions around peak time availability. Clarifications need to be sought 
Tier 1 to Tier 3 support issue response time. 

Negotiation Question 

The following questions were noted for the vendor: 

1. Need classification accuracy for a larger sample size 

2. Reports to track service usage needs to be defined 

3. The frequency and process for penetration testing to be clarified 

4. Application performance monitoring should be elastic 

5. Schedule to include UAT bug fixes timeframe and to incorporate tim 

6. Project Management and technical resources to be clarified. 

7. SLA Availability and SLA terms and conditions to be negotiate^ 


Overall ranking for Substantial Conformity 

Based on the discussions among the evaluation team me 
following ranking for Substantial conformity to specification! 

1. Amplify Education Inc. 

2. Istation 

3. Curriculum Associates 

4. NWEA 



unanimously agreed with the 


2. RFP Desired Specification 


The RFP desired specification 
discussions - 



four vendors. The following is the summary from the 


Amplify: 


The team was uncertain about the ability to upload evidence of learning. This appeared to be a negotiation item 
and was not included as part of the proposed package. The team was split on Amplify’s ability to incorporate a 
personalized blended approach to assessment and learning to meet the demands of diverse student populations. 


The team liked th'b online and observation options that Amplify offered. They were also satisfied with Amplify’s 
ability to assesses reading behaviors and print concepts of connected text. 

Associates 

members felt that the RFP response did not include clear measurement of print awareness for young 
hildren. They also agreed that this service did not provide the ability to upload evidence of student learning. 

The team liked the easy to read reports and adaptive assessment. 
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Istation: 

The team was concerned about the capability of the system to provide personalized blended approach to meet 
the demands of diverse student population. Oral language assessment was unclear or missing. The vendor 
reported working on ways to use voice recognition for oral language and no specific test is yet developed. Fr 
page 8 of the RFP response, "Students’ expressive language knowledge is captured by their ability to ide 
rhyming word from an orally given target word", this was not convincing for the voting members. The R 
response also did not specify about constructed response type assessment features. There was also u 
for some members about the availability of touchscreen availability for students which was clarified by the 
technical team member as being available. 

The team liked the adaptive nature of the assessment and how ISIP adjusts to each student’s tri 
early literacy to provide more accurate assessments and targeted, personalized instruction. 

NWEA 

According to the RFP response, Expressive language is "not directly assessed. With the inclusion of audio on our 
grades K-1 tests, we can assess more of the receptive components of oral language, including grammar, 
vocabulary, and syntax. Additionally, our grades K-1 assessment measure speaking and listening standards 
through questions about text read aloud and describing people, places, things, and events." Based on this 
response, the team had concerns about the capability of the system to provide personalized blended approach to 
meet the demands of diverse student population. Teaching modalities were not specifically stated in the RFP. 

The team was concerned about the vendor not providing response time directly but stating that they are willing to 
negotiate. 

The team liked the adaptive nature of the MA 





Summary 

After discussion, the team voted on Desired Specificatidh and the ranking is listed below: 

1. Amplify Education Inc. 

2. Istation 

3. Curriculum Associ 

4. NWEA 


3. Proof of Concept/Demonstration 

The Proof of Concept/Demonstration was ranked for all the four vendors. The vendors were ranked for the 
following three questions: 

1. Demo Script Adherence 

2. Ability to meet RFP Specification 

3. Ability to meet legislated timeline for implementation 

lify Education Inc. 

e voting members were unanimous in agreement that Amplify’s demonstration adhered to the demo script and 
expressed confidence in the vendor’s ability to meet the RFP Specification. It was pointed out that if at all any 
ith this vendor, the measures have to be scaled down to make it easier for educators and students. The team 
confident that with the customizations that will be needed to scale back measures, the service can be 
implemented statewide for the 2019 - 2020 School Year. 
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Curriculum Associates 

The team was in agreement that some of i-Ready’s processes demonstrated at the meeting was robust. 

However, they were split in their votes about i-Ready’s adherence to the demo script. The team voted 6 Yes and 
5 No for i-Ready’s demo script adherence. i-Ready’s oral reading fluency measure is currently not available, 
the vendor had indicated that this measure will not be available until the 2021 School Year. During the 
demonstrations, the vendor tied the rapid naming which is needed to screen dyslexia to the fluency mee 
Some of the voting members pointed out that rapid naming could be accomplished without WPM fluency 
measure. Some newer research uses object naming and number naming as a predictor. 

In i-Ready’s ability to meet the RFP specification one of the voting members voted Yes. However,The rest' 
a No on their ability to meet the RFP specification. 

The team unanimously voted all “No” on i-Ready’s ability to implement for the 2019 School yean^cause the 
vendor’s proposal and subsequent clarifications followed a districtwide implementation model. Based on DPI’s 
lessons learned in such a model, implementation requires more resources and more time. Also based ornhe 
vendor’s clarification response, to achieve the 2019 School Year implementation at a minimum, the state will have 
to plan to supplement Fluency measures and work with the vendor to update the SAML intlgration capabilities 
and add additional regional and statewide roles. These enhancements with a district implementatron model make 
implementation for 2019 School Year questionable. 

Istation 

The voting members agreed that Istation’s demonstration showed that they had robust reporting capabilities. 

The team was split on Istation’s adherence to the demo scripts and voted 5 Yes and 6 No. Oral language is a 
new assessment; text fluency and oral language are not a part of the overall ability score. There is no pure 
measure of letter knowledge. Fluency is incorporated in MAZE and not included in the overall score. 

The team voted 4 in favor and 7 against Istation’s ability to meet the RFP specifications. 



In their vendor demonstration clarification response, Istation mentioned that they have engaged with a third-party 
auditor to provide SOC 2 Type II attestation of their software application. This attestation is expected to be 
completed in a matter of months. This led to the question of delaying the approval of vendor award 
recommendation should this vendor be chosen for award. This had to be considered for the overall 
implementation timeline. The team unanimously agreed that this service can be implemented as such for 2019 
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NWEA: 


The team voted 6 in favor and 5 against NWEA's adherence to demo script. The key discussion point was the 
district implementation model and district by district rostering that was presented at the demo. The team felt that 
the demo was unclear on some of the activities that students complete for each measure. During the demo it wa s 
mentioned that MAP assessment is still being studied on the suitability as dyslexia screener. Also, there w^ 
disconnect between the standards and the student questions during demo. Text complexity was mentior 
during demo, but it was stated that only Lexile was used for this. Clusters were incorrect in several places ip the 
demo. 

Second graders can be benchmarked in different ways using K-2 or 2-5. There has to be appi 
place for second grade assessment as the student should be in the same test for the calendar 
considered, that the 2 - 5 level does not dip down to the foundational skills if needed. 

Progress Monitoring is currently under development and being validated. MAP readinUfluency for Progress 
Monitoring can be on demand. The biggest concern of the voting members were that the student transfer between 
districts is a manual process and would require upto 48 hours. The team unanimously voted “No” for NWEA's 
ability to meet the RFP specifications. They also voted 1 in favor and 10 against NWEA in their ability to meet the 
legislatively mandated deadline of 2019 School Year. 




After this the team ranked the vendors: 

1. Amplify Education Inc. 

2. Istation 

3. Curriculum Associates & NWEA (tied) 


4. Vendor Cost Proposal 

The Total Cost of Ownership (TCO) was determined for all the four vendors. This included the proposed cost and 
any additional costs indicated in the proposal. Certain vendors proposed alternate costs. 


Accordingly, the TCO for each of the vendors is tabulated below: 


Vendor 

Amplify 

Curriculum 

Associates 

Istation 

NWEA 

Low Cost per student 

$15.36 

$8.08 

$6.60 

$21.14 

High Cost per Student 

$25.78 

$22.48 

$6.60 

$21.14 

Rank 

4 

3 

1 

2 


The following assumptions were made in determining TCO: 

1. In the case of Amplify the higher cost was chosen because the higher cost option had the appropriate 
professional development component. 

Z In the case of Curriculum Associates the higher of the two cost was chosen because this cost included the 
Assessment and Instruction component. 

In the case of Amplify, a total of 20,000 kits were assumed at $125 per kit. This cost was distributed for three 



case of Curriculum Associates, in the demonstration clarification document with the statement that 
fluency will be offered from another vendor with no additional cost. There was also another statement to the 
effect “If we are selected for award, we understand there are many details to work out with NCDPI — not least 
of which will be cost. We anticipate the third-party fluency assessment purchase would be in line with 


RtAD-Consensus Meeting_11192018 & 11202018 


Page 28 of 31 




















Meeting Agenda & Summary 


currently available offerings, in the approximate $1/student/year range.”. An additional $1 per student was 
added to account for fluency. 


5. 


6 . 


Since the solution required a headset, the business team decided that each device would require 4 headsets. 
TCO was arrived at assuming 26,000 RtA devices and $10 per headset. 

Istation indicated that “While Istation recommends that students use headsets to reduce distractions du[ 
the assessment, they are not required”, the business team decided that each device would require 4 
headsets. TCO was arrived at assuming 26,000 RtA devices and $10 per headset 
NWEA indicated that “MAP reading fluency requires each student to use an over-ear headset with a I 
style microphone.* Districts and Schools will be responsible for purchasing and providing these to studdf 
Built in computer microphones and microphones in-line on a headset cord are not supports 
administration of MAP reading fluency.” The business team decided that each device wouf 
headsets. TCO was arrived at assuming 26,000 RtA devices and $50 per headset. 


For cost ranking as indicated above the vendors were ranked as follows : 

1. Istation 

2. NWEA 

3. Curriculum Associates 

4. Amplify Education Inc. 


5. Vendor Relevant Experience and Reference Checks 


Constance Bridges walked the team through her referenc 
NCDPI reached out to every reference at least 3 times. 



Per the procurement guidelines, 


Three reference check responses were received for Amplify, Curriculum Associates and NWEA. Two of the three 
Istation references responded. All the references said the vendors have been highly responsive and have 
addressed all issues. All the respondents agreed that they wilnrenew the vendor’s contract and will refer them. 


With this information, the voting membe 
Amplify, Curriculum Associates, ISKME 

6. Vendor Financial Sta 




e vendors the same for Reference Checks, 
were ranked 1 for reference checks. 


Meera Phaltankar walked the team through the results of her Financial Analysis. She said all of the four vendors 
have good liquidity positions and she did not see any going concerns. Unqualified auditor’s report was available 
for Curriculum Associates, Istation and NWEA. She was able to determine the quick ratio. Since the unqualified 
auditor’s report was not available for Amplify she could not do the quick ratio. 


Meera recorngienaed that Curriculum Associates, Istation and NWEA are ranked number 1 and Amplify be 
ranked number 4 for financial stability. 


otiaf^^Question: If NCDPI proceeds further with Amplify the company’s unqualified auditor’s report should 
ceivecW 



ed on Meera’s recommendation, the team ranked Curriculum Associates, Istation and NWEA as number 1 
Amplify number 4 for Financial Stability. 
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Meeting Agenda & Summary 


Final Ranking 

The team completed ranking of the four vendors and the outcomes are given below - 


Consensus Meeting 


Vendors 

1. Substantial 
conformity to 
solicitation 
specification 

2. RtAD SaaS 

Desired 

Specifications 

3. Proof of 
Concept/ 
Demo 

4. 

Vendor 

Cost 

Proposal 

5 Strength 
of 

References 

6 - A 

Vendor 

Financial 

Stability 

Phase 

Rank 

Order 


Rank 

Rank 

Rank 

Rank 

Rank^^ 

Rank 

Amplify 
Education Inc. 

1 

1 

1 

4 

1 

4 

► i 

Curriculum 

Associates 

3 

3 

3 


1 

^1 

3 

Istation 

2 

2 

2 

jit 


1 

2 

NWEA 

4 

4 

3 

\\ A 

1^ 

1 

4 


After this the team deliberated the merits of the services re' 
ranking. In summary, the team expressed unanimous agre 



marized below and reconfirmed the 
ranking outcome above. 


In the case of Amplify Education Inc., the service provides online assessment as well as observational 
assessment. Online assessment is available for students with appropriate self-regulation and computer skills; 
teachers continue to have the option to directly assess/observe students who are in need of regulation. This is 
especially critical because this read to achieve solution is expected to assess the pre-reading and reading 
behavior of students. The target population for this assessment is K-3 students. The students may come from 
different socio-economic background and ethnicity. The core measures of DIBELS have always been recognized 
as valid and reliable screener for risjf factors for dyslexia. This will satisfy the needs of HB 149 without 
overburdening the School DMncts to develop or identify additional tools for dyslexia screening. 

The online assessment takes about 17 minute's and the observation assessment about 12 minutes. However it 
has be remembered that the observational assessment takes 12 minutes per child and the online assessment can 
be group assessed. 


This assessm 



ilable offline. 


The issue of using developers in Ukraine for coding should be further discussed with DIT and Legal. Further 
clarificatiot^^needed from the vendor including identifying all associated risks. 

In the case of Istation who came second in ranking oral language is a new assessment; text fluency and oral 
language are not a part of the overall ability score. There is no pure measure of letter knowledge. This service 
does not satisfy the needs of HB 149 to act as a valid and reliable screener for dyslexia. Consideration should be 
given to how much the teachers may be overburdened by using different assessments for dyslexia and the read 
to achieve legislation. The school districts could be challenged to develop or identify tool to satisfy the dyslexia 
legislation. The assessment is also not diagnostic in nature in that it expects teachers to be diagnosticians. Also, 
this assessment takes about 40 minutes for assessment and progress monitoring. 
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Meeting Agenda & Summary 


The service allows recording students and playing back. The team also agreed that most schools have at least 
about 2 computers (Chrome book and iPad) per class. 


Additionally, the parent letters that are required to be sent home for the child requires manual entry of student 
data and this time should also be taken into account while considering the instructional time taken away from 
teachers. 


Next Steps: 

The budget bill authorizes the State Superintendent to supervise and approve the vendor selection, 
by the budget bill, the business owners will collaborate with the State Superintendent to inform hinr 
ranking and understand his priority. The meeting was adjourned and the evaluation team was notifies 
will be notified of the next steps after meeting with the State Superintendent. 



Action items resulting from the meeting follow. 



Action Items 


Item 

Assignee 

Due Date 

Status ^ 

Confirm the process with the 
State Superintendent 

Tymica 

12-5-18 


Notify the evaluation team 
about the next steps based on 
guidance received from 
Procurement 

Sri 

12-15-18 
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Software a 
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ostics 
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valuation 
eeting 


Dr. Pam Shue & Dr. Amy Jablonski 
October 5, 2018 1:00PM 






Agenda 


1. Introduction 

2. Project Background and 

3. Evaluation Team Co 

4. Evaluation Ground RTJl 

5. Proposal Evalu; " ~;ocess 

6. Reminders to e uccess 

7. Accessing Eval Team Site 

8. Questions 

9. Wrap i 



Jp 
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Project Background 



The purpose of this project is to 
competitive bidding process 
Session Law 2018-5; by th 


Superintendent (OSS) t< 
for a formative, diag 
Read to Achieve 



a 
in 

of the State 
e best solution(s) 
sessment for the 


obliaations outlj 

necyr 

i NC House Bill 149 to 

screen students 

Lfoia 

vslexia in arades K, 1,2,3 
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Project Background /> 

North Carolina state law requires kindergarten 
through third grade students to be |r^essed with 
valid, reliable, formative andjifiagriostic reading 
assessments. NCDPI is obfigated to provide these 
developmental^ appropriate assessments. 

Further pursuant to state law, the solution must 
assess student progress, diagnose difficulties, 
inform instruction and remediation, and yield data 
that can be used with the Education Value-Added 
Assessment System (EVAAS). 
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Project Background 

The assessments should also suppojjp- 

• Multi-Tiered System of ! S*jpppft^ 
Specific Learning disability Policy 


Dyslexia 


4 - 
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RFP Evaluation Objectiye 

To complete the RFP evaluation a?Kl4elect the 
finalist by November 14 th , 2018 ikorder to 
conduct further negotiation and award the 
contract on or before January 31 st , 2019. 
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Evaluation Team Compositio 


The evaluation team is composed o^^/ 


“ 


Voting Members 
Non-Voting Members (who are Subject Matter 





Experts) 

The specific role of 
in the following 


quisites: All Evali 




these groups are discussed 


Prerequisites: All Evaluation team members must sign and submit the confidentiality and conflict of interest forms. 
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Evaluation Team -Voting Membe 


Evaluation Team 
Member 

Pond, Karl 
Jablonski, Amy 
Shue, Pamela 
Berry, Erika 
Gossage, Chloe 


Organization 

NCDPI Enterprise Data & Reporting 

NCDPI Integrated Academic & Behavior 
Systems 

NCDPI Office of State Superintendent 
NCDPI Office of State Superintendent 
NCDPI Office of State Superintendent 



Role 

Voting Member Enterprise Data Mai* 

Voting Member Director, Integrated Academic and Behavior Systems 
Voting Member Associate Superintendent for Early Childhood Education 
Voting Membei Senior Policy Advisor 


Belcastro, Rebecca NCDPI K-3 Literacy 
Whitford, Abbey NCDPI K-3 Literacy 


Voting Membe 
Voting Member K 



egy Officer 
teracy, Piedmont-Triad Consultant 


Laney, Susan 

Loeser, Lynne 

Day, Kristi 

Parrish, Tonia 
Johnson, Mia 

Karkee, Thakur 
Dewey, Cynthia 

Hoskins, Matt 


NCDPI Integrated Academic & Behavior 
Systems 


NCDPI Exceptional Children 


NCDPI K-12 Standards, 
Instruction 
NCDPI K-3 Literacy 
NCDPI K-3 Literacy 




) Member K-3 Literacy Northeast Consultant 



t3er Integrated Academic and Behavior Systems Consultant, 
Research and Evaluation Specialist 

.. , Statewide Consultant for Specific Learning Disabilities and 

ing Member ADHD K a 

Voting Member Interim Section Chief for ELA 

Voting Member K-3 Literacy Consultant 
Voting Member K-3 Literacy Consultant 

Voting Member Psychometrician 
Voting Member K-3 Education Consultant 

rated Academic & Behavior .. .. .. , Integrated Academic and Behavior Systems Consultant, 

Voting Member Res “ arch and Eva|uation Lea(J Cons ' ltant 


NCDPI Accountability Services 
NCDPI Office of Early Learning 


Il'iEU P rerec f ui s'tes: All Evaluation team members must sign and submit the confidentiality and conflict of interest forms. 
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Evaluation Team - Non-Voting Mem 


Evaluation Team Member Organization 

Dunn,Tymica NCDPI Purchasing 


Lowe, Linda 
Viswanathan, Srirekha 
Snider, Eric 
AlHour, Julien 
Hunt, KC 

Phaltankar, Meera 

Hodge, Gin 

Lanier, Claudia 
Moates, Courtney 
Cantey, Joy T 
Reap-Klosty, Darlene 
Anselmo, Giancarlo 
Roberts, Amy 
Wilkes, Deborah 
Cooper, Shaunda 


Tomberlin, Thomas 


Pilonieta, Paola 



NCDPI Technology Services 
NCDPI Technology Services 
NCDPI State Board of Education 
NCDPI Technology Services 
NCDPI Technology Services 

NCDPI Financial Services 

Buncombe County Schools, 
Instructional Coach 

NCDPI K-3 Literacy 

New Hanover County Schools 

Guilford County Schools 

Chatham County Sc 

Cleveland County Schools 

Cabarrus County Schools 

Cumberland County Schools 

NCDPI Office of Charter Schools 

PI School Research, Data and 

g 



Role 

Non-Voting SME Procurement Spe 
Non-Voting SMEPMO Manager 
Non-Voting SME Project Manager 
Non-Voting SME Attorney 
Non-Voting SME Director, Technology Services 
Non-Voting SME Information Security Officer 

Non-Voting SME Director, Financial Services 
on-Voting SME County Instructional Coach 

X 

pting SME K-3 Literacy, North Central Regional Consultant 
ng SMEMTSS Instruction Specialist 
-Voting SME Director of K-12 Literacy 

on-Voting SMEMTSS Instructional Program Facilitator 
Non-Voting SME School Psychologist 
Non-Voting SME 

Non-Voting SMEESL Coordinator 
Non-Voting SME Education Consultant 

Non-Voting SME Director 

Associate Professor, Coordinator of the Undergraduate 
Non-Voting SME Reading Program 


II'IEU P rerec i uis ' tes: AI1 Evaluation team members must sign and submit the confidentiality and conflict of interest forms. 
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Evaluation Voting Team Member R 



ling evaluations 


Consists of representatives from DPI who are r< 

participate in all evaluation meetings for the entir< 

evaluation process from start to finish: 

• Review RFP objectives prior to begii 

• Participate in demos/orals 

• Review each responsive proposal and record strengths, 
weaknesses and clarification questions 

• Notify Project Manager of Clarification questions or concerns 
that arise 

• Through team consensus based on proposal review and 
demo, rank each proposal relative to other proposals to 
determine Finalists (short list) 

• Participate Best And Final Offers (BAFOs) 

SelecMnd Recommend Vendor(s) for contract award 


Il'ilWJ Prerequisites: All Evaluation team members must sign and submit the confidentiality and conflict of interest forms. -| 0 
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Evaluation Non-Voting Team Member 



Consists of representatives from DPI, LEA and 
supplement knowledge and provide feedback to Evaluation 
Team. The evaluation team will require advisors skilled in a 
variety of technical fields. 

• Review business, legal, technical, security; project 
management and procurement aspects of proposals 

• Ensure project schedule is adhered to. 

• Review financial statements to determine level of financial risk 
(high, medium, low) 

• Provide guidance on cost evaluation 

Provide inputs on business, technical, financial, reference 
check and project management aspects of the RFP. 


4S 



II'IEU P rerec l ui ^ es: AI1 Evaluation team members must sign and submit the confidentiality and conflict of interest forms. -| -| 
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RFP Evaluation Ground Rul 

Contact outside of Evaluation Team 

_ Procurement initiates ALL communication to/from vendors 

_ No discussion permitted with co-workers , managers , farrniy memoers or anyone 
else outside of the Evaluation Team (unless authorised and signed Conflict of 
Interest and Disclosure and Confidentiality Agreement forms are on file) 

- Do not speak to any vendor about the RFP, response^ or the selection process 

_ All questions and clarification points th^mfee throughout the process must go 
through Procurement Specialist 

>h contefemith Vendor(s) and make 
webinars, face-to-face meetings, etc. as 


Procurement Specialist will estal 
arrangements for conference ca 
appropriate 

Proposals must be treated as^ fifi 




ntial and proprietary 


Proposals and evaluation team materials including any portable storage devices, 
must remain locked and secure when you are not reviewing them 

Evaluation Team members should refrain from sending email messages that 
contain proposal information, ranking or any other information that must remain 
confidential. An%plarifications can be posted in the individual folder assigned in 
ShareEoiWmd email sent to the Project Manager. 
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General 


for EvaJji nation 

Sc 


Public Record; 

• As an Evaluation Committee member you are accountable for eyerytfung you write and do 
regarding the RFP, each Proposal and the evaluation process. D roposal evaluations are 
part of the RFP and contract files, and as such, are public records 


including the names of 


the Evaluators. 


Proposers may request to review evaluations of all D roposals may use the information to 
submit a protest. In addition, Proposers are entitled to ask for a debriefing and Evaluation 
Committee members could be required Jo attend and explain scoring. 


Any member of the public may als 
process in compliance with North Ca 

Written comments will be disclo 

Please do not transmit 
clarifications, please se 
contact you to fincLout m 



In SharePoint 
questions^ ck 




review all documents relating to the RFP 
blic records law. 


y requesting party as part of the public record. 

tial proposal related details via email. If you need 
ail to the Project Manager and the Project Manager will 



valuator has a folder assigned to them. Please upload a document with 
ations and send a SharePoint message to the Project Manager. 
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Proposal Evaluation Proce 



b. 

c. 

d. 

e. 

f. 

g- 



1. Proposal Opening 

2. Check for Initial Purchasing Office Review Response 

3. Kick-Off Meeting ★ 

4. Proposal Evaluation - Phase 1 (Competitive 

a. Vendor Demos ( record demo script & ask gueffions ) 

Individual Review & Recording Strengths. Weaknesses, Clarifications 
Team Review of Compiled S,W,C ; Irlftial Consensus Ranking 
Formal Clarification Questions to Purchasing for Vendor Responses 
Conduct In-depth Reference Checks 
i. Assess K3 Solution Relevance & Experience 
Consider Total Cost of Ownership, Relevance, Confidence 

Team consensus and ‘Short List” Finalists for BAFO 


Best And Final 


6. Award Recomme 




vFOs) - Phase 2 
n Package 
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RFP Process Completed so 






sals = 


1. Bid Opening on 10/02/2018 

2. The following bidders have sent thdi 

Amplify Education Inc. 

Curriculum Associates 
Imagination Station Inc. 
iv. NWEA 

3. Initial responsiveness e' 

Procurement - 

Amplify Education 
Curriculum Associate 
Imagination Statiorlint 
iv. NWE/ 

4. The responifive bidder response has been published 
in SharePoimt. 


update from 
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RFP Evaluation Criteria 



The evaluation criteria included in the RFP is ii§le#below. 
The criteria at the top of the list are relative, 
important than those at the bottom of the Hst. 

aV 

1 .Substantial Conformity to Solicitation Specifications - Refer 
Attachment A ; Tables A, B, C, D and E 

2. RFP Desired Specification - Refer Attachment A Table F 

3. Proof of Concept/Demonstration - Responsive Vendors 

4. Vendor Cost Proposal - Refer Attachment A Table G 

ability - 


Paragraph 14. 
6.Vendor Financial Sta 


- Refer Section V Paragraph 3 
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RtAD RFP proposed Evaluation 

in scope 

There will be additional Conference call meetings a< needed 

Kick-off Meeting 10/05/18 1:00pm-3:00pm ^ DPI - Room 504 


Vendor Demonstrations 10/22/18 & 10/23/18 


Consensus Meeting 


8am - 


NCDPI - State Board 
Room (7 th floor) 


Best and Final Offer 
(BAFO) 




11/19/18 & 11/20/18 


From 11/26/18 to 12 


NCDPI - Room 504 
(11/19) & State Board 
Room (7 th Room) 

NCDPI - Conference 
Calls 


review feedback no later than October 


Return the Demo S< 

10 th . 

Please complete your review of proposals for consensus meeting by 
November 8 th to allow time for compilation of results. 
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Proposal Evaluation - Phase 


Vendor Demonstrations 


Vendor demonstration scripts will be shared with all evalua 
Please review and provide your feedback no later tha 


,9 

lalors on 1 0/8/1 
)/10tf8 


Each Evaluation Team Member and Non-votin 
demonstrations and Vendor presenters to do 
weaknesses of the solution and team. 

Tymica Dunn will issue a Clarification 
demonstrations to obtain written docu 

Evaluation Team members can lev 
solution(s) as needed, trial peri 
customer tools, user guides and 





sely observe product 
ional strengths and 


endors the day after the onsite 
f the demo session. 

ay trial licenses to further assess 
access to proposed solution features, 
materials 






20 


a r Public Schools of North Carolina 






Individual Proposal Evaluat 

All responsive proposals are available in the 
Evaluation SharePoint. 



Evaluation Committee Members 

- Read the RFP and all Addenda. 

. Read each Proposal and indgpl 
questions in the checklist. 


. Include strengths and 
and vendor clarificatio 

- Evaluate Propos 
vendor clarificatio 


. Complete 
the Eval 




,v . 


ly review and respond to 


ibserved during demonstrations 
ther your review feedback. 

only on the responses in the RFP and 


ecklist provided by 11/8/2018 in preparation for 
Committee Consensus meeting. 
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Rationale for Scoring 

• Each evaluator will be provided an excel 
document their feedback in a consistent 

• In the checklist, the specifications are listed as recorded in 
the RFP. 

• In the column ‘Meets Requirerrr"^ ~ 

- Yes indicates that the vendor has acjj ^Sssed the specification and 
the evaluator is satisfied. \.^ 

■not addressed the specification in 




- No indicates that the vem 
the RFP Response 

- MayBe indicates that the response is unclear and that the 
evaluator needs further clarifications 

Document Sfrangths and Weakness of each specification 
in their aooropnate columns. 

Any ambkrtfities should be noted down in the Clarification 
.mJj 
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Consensus Meeting on November! 9 th 

and 20 th 


After all the Evaluation Committee members nave completed 
their checklist, the Evaluation Committe^fl/illjPieet to jointly 


discuss the merits of each Proposal 




It is not necessary that the voti 
point, however, this meeting 
Committee members to di 
SMEs and ideally reacfr&xjr 



irs concur on any given 
lortunity for Evaluation 
a group with input from 
agreement to rank the vendors. 


Based on the ranking'during the consensus meeting the finalist 
vendors will be selected for Best and Final Offers (BAFO) 
negotiation. 
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Team Consensus Ranking Methodo 





oposals pay 
ings relative to 


Team Consensus Ranking, Clarifications & Finalist Sel 

• Support for cost analysis will be provided, as yo 
attention to areas that may increase cost or result 
other proposals 

• Outcomes from In-depth Reference Check^faiN be provided to the 
Evaluation Team 

• The Project Manager will compile all recorded strengths and 
weaknesses and validate them against vendor clarification responses 
where appropriate before the team meets for consensus ranking 

• Evaluation Team Members and Non-voting SMEs will review the 
compiled strengths, wef knesse£ cost analysis and clarifications 

• Everyone will meet and the voting members will conduct a consensus 
ranking to make recommendations for vendor finalist(s) for live product 
demonstrate 


Timely a 

notes a 



rom the voting members on the consensus meeting 
stration script is very important. 
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Best And Final Offers (BAFOs) - Ph 







schedule, 


Purpose of BAFO step is to: 

• allow bid offerors to revise their offers; revisions may apply 
technical requirements or other terms of the propose( 

• respond to any errata in the vendor’s proposal 

• obtain the Vendor’s best and final cost offer 

Evaluation Team will narrow Finalist list down before beginning the BAFO 
process to preferred and possibly second preferred Vendor 

Procurement Specialist will coordinate Legal reviews as appropriate 

Negotiation meetings are allowed during BAFO and when the committee and 
the evaluation team is comfortable, a single BAFO meeting is conducted to 
finalize discussions and obtain approval. 
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Award Recommendation Packa 


Evaluation Team and Project Manager prepares Award Recommendation 
Package with supporting documentation to justify the best value decision. 

Award recommendation with supporting details will be presented by Project 
Manager to the Voting Members and DPI Leadership for review and approval. 
Upon approval, the Project Manager sub 
package to the Procurement Specialist a 

The Procurement Specialist prepares 


supporting details and submits to 





recommendation 
ounsel. 

Recommendation Letter with 
mission to award the contract(s). 
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Reminders to Ensure Suo 


& 

Failure to adhere to the ground rules may 

& 


compromise the entire RFP 

If you have any questions 
permissible, please conta 


The less information 
Evaluation Team ( 
decision-makina Dr 





is and is not 
oject Manager. 


with those not on the 
rs required to support the 
15) the better 

A document with contact information has been 
posted in the project SharePoint repository. Please 
provide your contact information including alternate 
phoneTiumbers to call or text incase of need. 
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RFP Deliverables (part of public record) 


1. 

2 . 

3. 

4. 

5. 

6 . 

7. 

8 . 

9. 

10 . 
11 . 




he 


: orms by all 


Request for Proposal and Responses 

Evaluation Team (Names, Business Title and 
evaluation) 

Confidentiality Agreement and Non-Qisciosurt 
evaluation team members 

Kickoff Meeting Presentation ai 
Evaluation Checklists 
Team Consensus Rankii 

Meeting Minutes from the Consensus Meetings ( approvals from 
all voting members are required on these documents) 

Demonstration Script 

Demonstration Vendor Clarification 

Best and Final Offer documentation 

Contract Award Recommendation Document 
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Evaluation Document Repo^ sitor 

Evaluation Team Members have been granted 
access to the SharePoint site,, v Nr 


Each member has an assi 
name to which only the 

For non DPI users p 
credentials to logi 



der with their 
r has access. 


se your Microsoft 
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Evaluation Document 


Read to Achieve RFP 


Home I NCDPI I DPI Team Sites I DPI Internal I DPI External 


This is a collaboration site for Read to Achieve RFP Evaluation. Access is resisted only ti 

Next Meeting on January 19th between 1.30 and 3.30 PM 


RFP Responses 

Type Name Modified By 

There are no : tems to show in this view of the "RFP Responses" 
document ’ibrary. To add a new item, dick "Add document’. 

+ Add document 


RFP Evaluation Doc 

Documents 

uments 


□ Type 


fc^dned 

Modified By 


Larie^^ 

l/!c9018^*: PM 

Srirekha Viswanathan 

Amy 

./KV ’ V 

1/16/2018 9:36 PM 

Snrekha Viswanathan 


srM 

1/16/2018 9:48 PM 

Srirekha Viswanathan 


^hloe 

Gossage 

1/16/2018 9:40 PM 

Snrekha Viswanathan 


Cynthia 

Dewey 

1/16/2018 9:49 PM 

Srirekha Viswanathan 

m 

Dave 

1/16/2018 9:47 PM 

Srirekha Viswarathan 


Search this site 


Achieve Evalution Team Members. 


RFP and Addendum 


Type 

Name 

Modified By 

£ 

4Q-IT00115-18 Read to Achieve NextGen RFP 
Addendum Cl 

Srirekha Viswanathan 

S 

40-IT00115-18 Read to Achieve NextGen RFP 

Posted 12-6-2017 

Srirekha Viswanathan 

+ Add document 





Team Communication 

V Q Name Modified Modified By 

RtAD ... 15 hours ago Virginia, hodge@bcserraii.org 

NextGen 

Evaluation 

Team 
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Evaluation Checklist. 'A 

v'v 

Each evaluator has a workbook per vendor in 
their assigned folder. 

Drkshec 


Each workbook has two 


1. Substantial Confo 


2. Desired Specifi 

Please document 
clarificatio 
propos 




leets - 


uation 


valuation 

>e strengths, weakness and 
service as identified in the 
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Questions 



question 


quesUon 

i on 




J 
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Read to Achieve - Evaluation Checklist for Substantial Conformity to Technical Specifications 
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SlA-002 \ Describe the propose 
























































































































































































































































Read to Achieve - Evaluation Checklist for Desired Specification Evaluation 

Vendor :_ NWEA_ 

Evaluators :_ Non-Voting SMEs- 
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OS-009 

DS-010 

DS-011 

OS-012 

DS-013 

DS-014 

DS-008 

DS-007 

DS-006 

DS-005 

DS-004 

DS-003 

tndex 

DS-001 

DS-002 

Describe the open standards that can be used for interoperability with your 

service. 

!Describe other open standards (other than CEDS defined in Technical 

[Specification #20) that are used for interoperability. 

Describe how the proposed solution approaches print awareness for young 

children. 

Describe the proposed service's ability to maintain a portfolio of student's 

ongoing development over time. 

Describe the proposed service’s ability for authorized users to upload evidence of 

learning. 

Describe how the proposed solution includes a constructed response feature for 

responding to text dependent questions (NAEP) - 

https://nces.ed aov/nationsreportcard/ 

Describe how your solution provides communications to parents including the 

ability to generate strategies/tools for them to be able to help their children at 

home. Explain the research and vetting process for these recommendations. 

Online professional development options for teachers and administrators 

pertaining to the use of the assessment system and how to analyze and use the 

data to make informed instructional decisions for students. 

Electronic student, class, school, and district reports on assessment results to 
help all educators make instructional decisions based on the data, including a 

report that tracks student proqress/qrowth. 

The assessment system incorporates innovative and evidence-based 
approaches utilizing assessment results to assist in recommending instructional 
strategies. Describe the system's capability to provide on demand (real time) 
assessment data and instructional strategies recommendation. 

Incorporates a personalized blended approach to assessment and learning, 
including multiple teaching and assessing modals to meet the demands of 
diverse student populations with a wide range of learning needs. 

may be negotiated at that time. 

Support 3 to 5 second or less web page response times. However, if the vendor 

proposal is recommended by the evaluation team to the competitive range, all 

SLA terms may be negotiated at that time. 

Evaluation Specification - Desired Specification 

The Vendor’s RtAD SaaS Solution may provide touchscreen functionality to 

conduct student assessments on supported devices. 

Support at least 99.9% uptime availability. However, if the vendor proposal is 

recommended bv the evaluation team to the comDetitive range, all SLA terms 
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Read to Achieve - Evaluation Checklist for Desired Specification Evaluation 

Vendor : _Amplify Education Inc. _ 

Evaluators :_Non-Voting SMEs _ 

SMEl SME 2 SME3 SME 4 SME 5 SME 6 SME 7 SME 8 5ME 9 SME 10 SME 11 
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RtAD Vendor Demonstrations 
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I Contract Total 

Grand Total for the entire Contract |$24,575,445.75 



Vendor - NWEA 
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Number of Students 469,445 

Number of Teachers\/ 7 26,000 


Vendor - Curriculum Associates Low Cost 


One Time Cost 


Cost component 

One-Time Cost 

Des 

Project Management 

$0 


Customization required for 

implementation 

$ 0 

y 

Project Deliverables and User 

Documentation (Specify 
details) 

$0 

V 

Installation / Conversion / 

Migration / Implementation 
Costs 

$ 0 

Standard system integration, as 
in the i- Ready assessment sys 

Training including all training 
materials 

See Recurring Cost “Other" 
row 


Others (describe) 

See Recurring Cost “Other" 
row 


Total One-Time Costs 

$0 

X.X 

Cost component 

Recurring Costs 

Year 1 

Year 2 

Subscription fees or cost * 

$2,253,336.00 

$2,253,336.00 

Enhancement Cost 

$ ^ 

$ 

Technical Support / Customer 

Service 

1 

$ 

Others (describe) - 

Professional Development 
(Core Plan) 

$867,000 

$867,000 

Total Recurring Cost 

$3,120,336.00 

$3,120,336.00 


^ Grand Total fo 



Notes: 

1-Ready assessment system is $6/student/year for reading. Cost shown is for 1-Ready assessm 
Unlimited access to our Technical Support and Customer Service teams during business hours 
centralized onsite PD sessions in Years 1 and 2 (289 sessions each of two different course 
^^Uiagnostic administration and one delivered after. 289 centralized onsite PD sessions in Year 3 
28 centralized onsite Leadership Best Practices PD sessions per year will be free for the purch; 


Total Proposal Cost 
Total Contract Cost per year 
Total Contract Cost per Student 


$ 8 , 927 , 508.00 

$ 2 , 975 , 836.00 

$6.34 
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Vendor - Curriculum Associates High Cost 
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